Category: Coding

WebSockets from Ember, Rails, Nginx with AWS ElastiCache Redis

When we finally upgraded our Rails application to v5, plans were put in place to take advantage of its WebSockets capabilities.

Locally on development machines, things seem to chug along smoothly until it all broke down on our staging stack.

Here’s the basic setup:

* Rails 5.0.x
* Ember 2.x (using Ember Cable for WebSockets)
* Puma
* Nginx (for reverse proxying Ember calls to Rails + serving Ember app)
* AWS ElastiCache Redis
* Postgres hosted on AWS RDS
* EC2 instance + ELB + custom security group… etc.
* Entire stack is orchestrated with Chef running on AWS OpsWorks

I didn’t think the set up was anything out of the ordinary per se. The only exception is probably my choice of using ElastiCache instead of having an EC2 server running our own Redis instance (or having it hosted at Redis Labs or something). But other than that, it’s pretty vanilla.

Immediately we rant into trouble having Ember making a proper connection. We kept seeing this error:

WebSocket connection to 'ws://staging.domain.name/cable' failed: Connection closed before receiving a handshake response

First thing first, we had to make sure our /cable endpoint was proxied properly so that Rails can answer the calls. So in our main Nginx config file, we added:

1
2
3
4
5
6
7
8
9
10
location /cable {
    proxy_set_header   X-Real-IP $remote_addr;
    proxy_set_header   X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header   Host $http_host;
    proxy_set_header   Upgrade $http_upgrade;
    proxy_set_header   Connection "Upgrade";
    proxy_redirect     off;
    proxy_http_version 1.1;
    proxy_pass         http://app_server;
}

It still didn’t quite work.

1
2
3
4
5
I, [2017-03-22T18:46:39.166623 #4175]  INFO -- : Started GET "/cable" for 127.0.0.1 at 2017-03-22 18:46:39 +0000
I, [2017-03-22T18:46:39.167699 #4175]  INFO -- : Started GET "/cable/" [WebSocket] for 127.0.0.1 at 2017-03-22 18:46:39 +0000
E, [2017-03-22T18:46:39.167837 #4175] ERROR -- : Request origin not allowed: https://staging.domain.name
E, [2017-03-22T18:46:39.167950 #4175] ERROR -- : Failed to upgrade to WebSocket (REQUEST_METHOD: GET, HTTP_CONNECTION: Upgrade, HTTP_UPGRADE:)
I, [2017-03-22T18:46:39.168064 #4175]  INFO -- : Finished "/cable/" [WebSocket] for 127.0.0.1 at 2017-03-22 18:46:39 +0000

Obviously we need to specify in Rails that our Staging subdomain, along with its https protocol should be allowed. So we added that to our Rails /config/environments/staging.rb file.

config.action_cable.allowed_request_origins = %w(https://staging.domain.name staging.domain.name)

Once we squared that away, we were still left with that dreadful error on line #4 above.

Failed to upgrade to WebSocket (REQUEST_METHOD: GET, HTTP_CONNECTION: Upgrade, HTTP_UPGRADE:)

Somehow, the $http_upgrade variable in Nginx is simply not being set despite browser’s request. As it turns out, WebSocket traffic isn’t quite HTTP/HTTPS. The devil is in the details of the AWS ELB (load balancer).

I simply swapped out HTTPS with SSL. And BOOM! Like that, it worked:

Successfully upgraded to WebSocket (REQUEST_METHOD: GET, HTTP_CONNECTION: Upgrade, HTTP_UPGRADE: websocket)

Now connection was being made properly. Except for this one little thing: The browser had to keep reinitialize the connection every few seconds. And we noticed the UI simply wasn’t being updated despite changes to data. Sigh…

Next stop was to test if the EC2 instance itself was actually talking to AWS ElastiCache. There are a couple of ways to try this:

In Rails:

1
2
3
4
5
irb(main):001:0> redis = Redis.new(:host => 'my-elasticache-instance-identifier.0001.usw2.cache.amazonaws.com', :port => 6379)
=> #< redis client v3.3.3 for redis://my-elasticache-instance-identifier.0001.usw2.cache.amazonaws.com:6379/0>
irb(main):002:0> redis.ping
Redis::CannoConnectError: Error connecting to Redis on my-elasticache-instance-identifier.001.usw2.cache.amazonaws.com:6379 (Redis::TimeoutError)
...

Using Telnet:

1
2
3
$ telnet my-elasticache-instance-identifier.0001.usw2.cache.amazonaws.com 6379
Trying /ip ADDRESS/...
# eventually times out

OMFG. WHAT ELSE IS WRONG NOW??!!

As it turns out, the devil, again, is in the details. This time, it’s in AWS Security Groups.

AWS resources are only allowed to talk to each other if said resources are within the same security groups (among other things). In this case, AWS ElastiCache has a default security group, which we’re not allowed to change (not to my knowledge anyway). So the trick was to make sure that our OpsWorks Layers also have been assigned the default security group. Now, OpsWorks documentation says that you must reboot each instance within the layer for the new security group settings to kick in. But the truth is I had to spawn brand new instances for the new security group to stick. But once that was done, everything finally started to gel:

1
2
3
4
5
6
irb(main):001:0> redis = Redis.new(:host => 'my-elasticache-instance-identifier.0001.usw2.cache.amazonaws.com', :port => 6379)
=> #<redis client v3.3.3 for redis://my-elasticache-instance-identifier.0001.usw2.cache.amazonaws.com:6379/0>
irb(main):002:0> redis.ping
=> "PONG"
...
</redis>

I hate technology sometimes.

References:
* https://blog.jverkamp.com/2015/07/20/configuring-websockets-behind-an-aws-elb/ (cached in PDF)

Mac OSX Yosemite Automation with Javascript

In Apple’s latest Mac OSX update, a new programming language support was added on the OS level without much advertising — Javascript. But Apple did provide documentation about it. A couple of nice folks published great resources on how to leverage them.

It’s really amazing seeing how far Javascript has come in the past few years, penetrating practically the entire stack of web application development cycle. It’s an exciting time to be a software developer.

Ember.js Lessons Learned Retrospective

Ember.js LogoWhile googling for some syntax on Ember.js, I came across this incredibly helpful list of “lessons learned” from Landon Noss aptly named “Things I wish someone had told me when I was learning Ember.js“. He documented 28 tips learned from the battlefield. I love lists like that: Concise lessons distilled down to digestible bullet points, enough for others to go through the same troubles and speed up the learning curve. Kudos to Landon for the awesome list.

Just in case the article, the link or the site disappears, here’s a PDF version I generated for safekeeping.

Debugging Ruby 2.1.x and Rails 4.1.x with RubyMine

One of RubyMine’s strongest features is its debugging tool. But it’s always been finicky to get it working for me. I finally got it to work since upgrading to Ruby 2.1.x. The situation was complicated by Ruby 2.x not yet supported by debugger gem… etc.

To get everything to sing in harmony, here’s what I used:

And that was it!

Javascript Structs and ImmutableStructs

This is a pretty cool article on implementing structs and immutable properties & objects in Javascript. Data structure like this makes it possible to construct and serialize user roles and capabilities objects on the client-side user Javascript that prevents users from tempering with the view. Bad ass!

Here’s a PDF backup of the article just in case the site or URL goes blind. You can’t be too safe!

SuperHero.js for Javascript Best Practices

Javascript has been evolving way more quickly than I have been able to keep up. I’m still behind on some of the recommended best practices and modern techniques. Luckily someone had the good sense of putting up and documenting these things. I hope s/he/they keep up the good work to spare the rest of us.

This wonderful resource is SuperHero.js. It keeps up-to-date a list of resources on best practices and latest techniques in the world of Javascript for the rest of us peasant coders. Kudos to the site maintainer(s).

Duck Typing in Javascript

PHP was the first programming language I learned. But Ruby has been the language that I love. One of the patterns Ruby champions is the idea of Duck Typing. It’s a great way to avoid using if/else statements too excessively. In Javascript, this can be easily implemented as well. This article does a great job explaining it with some sample code. It’s a good thing!

Ruby’s #length vs #size vs #count Methods

Before googling for the article, I knew going in that .size is an alias to .length. But I didn’t know there was more to the story. And what about .count?

Here’s what Josh Susser wrote verbatim:

In Ruby, #length and #size are synonyms and both do the same thing: they tell you how many elements are in an array or hash. Technically #length is the method and #size is an alias to it.

In ActiveRecord, there are several ways to find out how many records are in an association, and there are some subtle differences in how they work.

* post.comments.count – Determine the number of elements with an SQL COUNT query. You can also specify conditions to count only a subset of the associated elements (e.g. :conditions => {:author_name => “josh”}). If you set up a counter cache on the association, #count will return that cached value instead of executing a new query.

* post.comments.length – This always loads the contents of the association into memory, then returns the number of elements loaded. Note that this won’t force an update if the association had been previously loaded and then new comments were created through another way (e.g. Comment.create(…) instead of post.comments.create(…)).

* post.comments.size – This works as a combination of the two previous options. If the collection has already been loaded, it will return its length just like calling #length. If it hasn’t been loaded yet, it’s like calling #count.

Retain Vim Color Scheme in sudo

Ever wonder why vim loses its color schemes and settings when you are in sudo mode? I hate it. But there’s a way to preserve your vim settings while in sudo mode.

sudo -E vim some_file.txt

The

1
-E

switch is the holy grail. Or you could use sudoedit if you have it set to using vim. And it’d have the same effect.

Resources for Working with Objective-C’s Core Data

I started dabbling with building an app in Objective-C and Xcode recently. And Core Data seems to be a nice way to work with persisting data across app launches. But its learning curve is notoriously steep (Apple’s own documentation doesn’t help at all).

But Techotopia has a nice and short overview of what Core Data is and how it works in a way that’s not intimidating. But the best source for me by far is “Core Data for iOS and OS X with Simon Allardice” from Lynda.com. I’m a visual/audio learner. And the best way for me to learn is usually through a Youtube video or some clips from Lynda’s web site.