Rails CDN and ActiveStorage

When serving assets and attachments from Rails and ActiveStorage, a CDN is a great way to reduce load on your web server and speed up content delivery to your users.

Asset CDN

First, even if you’re not using ActiveStorage, you’ll want to set up a CDN to stand in front of your app. This will proxy requests for images, fonts, javascripts, and stylesheets that live in your app’s repository.

The ideal setup here is to serve your assets from your app, set a high cache-expiration, and serve your assets through a CDN.

Setting up your Asset CDN

For a CDN, I recommend Amazon CloudFront. You’ll want to create a web distribution which points at the root of your app. Heroku has a good guide on that.

Once it’s set up, set an environment variable¹ (e.g. ASSET_HOST) to the distribution domain name and configure your asset_host. Also, set a long expiration for your assets. In config/environments/production.rb ^{2 3}:

# Tell our CDN and browser to cache assets for a year.
config.public_file_server.headers = { 'Cache-Control' => 'public, max-age=31536000' }

# Serve images, stylesheets, and javascripts from an asset server.
config.action_controller.asset_host = ENV['ASSET_HOST']

Once you’ve configured the above, your production assets should serve from your CloudFront CDN.

For more information, on serving assets, caching, and cache invalidation, see the Rails Guides.

ActiveStorage Attachments served through a CDN

When you serve attachments from a cloud storage service (I’m using S3), it will be coming from a different domain than your app. One solution to this would be to stream the file contents through your app.

I opted for creating a second CDN distribution that sits in front of the cloud storage provider that ActiveStorage serves for its 302 redirect ⁴. This does require a little configuration. Thankfully nothing needs to be monkey patched.

1. Create a second CloudFront web distribution that points to your S3 bucket and point your ATTACHMENT_HOST env variable at it (prepended with https://).
2. In lib/active_storage/service/s3_directory_service.rb, create the service in this gist.

3. In config/environment/production.rb, set ActiveStorage’s service_urls_expire_in :

   # Tell our CDN and browser to cache attachments for a year.
   config.active_storage.service_urls_expire_in = 1.year

4. In config/storage.yml, configure the service and default cache_control for uploaded attachments by adding the upload key ⁵:

   amazon:
     service: S3Directory
     # ...
     upload:
       cache_control: 'public, max-age=31536000'

5. When linking to the attachments, use the rails_blob_url helper like so: ⁶

   rails_blob_url user.avatar, host: ENV['ASSET_HOST']

   (For variants, you’ll need to call rails_representation_url instead.)

Now your ActiveStorage attachments will serve through the two CDNs: the first request will hit Rails’ Representations or Blob controller (through the Asset CDN) which will redirect to the service url for the attachment (through the Attachment CDN). If this attachment has been served to a previous user, no request will hit your web server as CloudFront will have it in its cache. If that user has requested the attachment before, it well be served from the browser’s memory and no request will be made.⁷

MacBook Pro Battery Drain

My main computer is my iMac. It seemed like every time I opened my MacBook Pro, it would show this screen:

So I researched a bit and fixed it. Here’s what I did:

• Turn off “PowerNap while on battery power”. This is the Energy preference pane.
• Turn off “Allow Bluetooth devices to wake this computer”. This is in the Bluetooth preference pane under Advanced.

That’s it.

Now, when I open my MacBook the battery has barely moved from the previous day. For example, 3 days ago I closed my laptop with 64% battery. I opened it this morning with 50% battery. A 5%/day battery drain is excellent for a computer that’s keeping its running memory actively powered. Previously I’d see what seemed to be 5%/hour (I didn’t actually measure it but rarely did I open my laptop and it wasn’t dead).

If you want to track your battery usage, check out iStat Menus. It has a graph to show battery drain over time (past hour, day, week, month).

Ping Tools in macOS

I was looking for an outbound ping tool that I could run continuously on a secondary monitor or in the background so I could quickly troubleshoot internet outages or packet loss.

For example, when I’m in a video call, if the other person starts to have issues with audio/video, I can, with a glance, tell if it’s me or not by looking for latency/packet loss amongst the hosts I’m pinging.

ICMPUtil is the best I’ve found. It’s $6 on the App Store.

I recommend setting it to use a 1 second ping interval with a 64 byte payload. 0.1 second pings look cool but many servers including google.com won’t respond to ICMP that quickly. The default payload of 32 bytes is half the default value for a typical ping command so results may differ slightly if you don’t use 64 bytes.

Add several hosts so if there’s an outage with your internet, you’ll see all hosts aren’t responding. If there’s just an issue with say Google or AWS then multiple hosts could fail but it’s not your internet.

Other Tools

Some other options:

• mtr - This is a traceroute terminal app that continually pings each host in a traceroute
• autoping - GUI mac app I used to use; no longer available though

Adding a Custom Postgres Type in Rails 4.2

In a Rails app at work, we are using a PostGIS data type of geometry and we were getting this warning in our logs:

unknown OID 16391: failed to recognize type of 'geo'. It will be treated as String.

This is because by default, Active Record doesn’t support the OID 16391 for this data type. We can add our own in Rails 4.1 as recommended by Rob Di Marco.

READMORE

In Rails 4.2, the OID module got reworked a bit and that solution doesn’t work anymore. Well Rob helps as again by recommending a new solution for Rails 4.2. The site isn’t loading for me so I pulled the code from Google Cache, cleaned it up a little and represent it to you here.

In config/initializers/add_custom_oid_types_to_active_record.rb:

# ActiveRecrod will `warn` (to stdout) when first connecting to the adapter:
#  unknown OID 16391: failed to recognize type of 'geo'. It will be treated as...

# Registering the `geometry` type (typically not supported by Active Record) will
# prevent this warning but AR will continue to treat it the same. This cleans up
# logs and test output.

ActiveRecord::ConnectionAdapters::PostgreSQLAdapter.class_eval do
  def initialize_type_map_with_postgres_oids mapping
    initialize_type_map_without_postgres_oids mapping
    register_class_with_limit mapping, 'geometry',
      ActiveRecord::ConnectionAdapters::PostgreSQLAdapter::OID::Integer
  end

  alias_method_chain :initialize_type_map, :postgres_oids
end

Configuring Pow including SSL

I used pow in the past and didn’t like it. I’ve given it a second chance and it seems to be working out. I’d like to give Invoker a try though but I’ll save that for another day.

Pow is a daemon that runs your rack (Rails) apps in the background when you request them at a special example_app.dev domain. You don’t have to start your app manually anymore (e.g. rails s). If you don’t use your app for 15 minutes it kills your app’s process to free up memory (restarting it on the next request). It doesn’t run workers (I recommend inline que workers via ActiveJob) or other daemons such as those in your Procfile. It just looks for a config.ru and (presumably) does a rackup on it. It’s very simple to set up and it’s Mac OS X only.

Install Pow

• Before running the install script, configure pow to use port 3300 rather than 80; this makes port forwarding easier and avoids unnecessary sudoing. You may prefer to skip this step or choose another port besides 3300.

echo 'export POW_DST_PORT=3300' >> ~/.powconfig

• Run the pow install script and configure pow to serve your app

curl get.pow.cx | sh
cd ~/.pow
ln -s ~/Code/example_co/example_app # Or where ever your repo is

• Ensure your application is running:

open http://example_app.dev:3300

Configure SSL with tunnelss

• Install tunnelss on your system:

gem install tunnelss

• Start tunnelss (I recommend 4430 as it does not require sudo):

tunnelss 4430 3300 # Change port 3300 if you changed DST_PORT (80 is default)

• Open the SSL URL to the app:

open https://example_app.dev:4430

• Trust the “unverified” self-signed certificate when prompted (should require admin password). Screenshot for Safari.

Adding the SSL certificate to iOS

• In your shell, run open ~/.tunnelss.
• Send cert.pem to yourself (email/iMessage).
• Open the attachment on your iOS device and install the certificate (should require your pass code and warn you about it not be a verified certificate).
• To access your application, use xip.io. On your mac run ipconfig getifaddr en0 to get your local IP to use in the xip.io URL (e.g. https://example\_app.10.0.1.4.xip.io:4430).

Note: If you’re testing remotely you will need an SSH or VPN tunnel.

Logs, restarting, pry and misc

• Logs can be viewed in log/development.log. A useful alias for this is:

alias devlogs='tail -f ~/Code/*/*/log/development.log' # May need to change path

• Restart the application using:

touch tmp/restart.txt # Will restart on next page load

As usual, Rails handles much of the code reloading for you so you won’t have to do this with every request. Just the typical reasons you’d need to restart the server for (modifying initializers, adding gems, modifying load paths, etc).

• Since pry isn’t connected to a tty, you won’t be able to start a debugger from your code the normal way (binding.pry). pry-remote has our back. Use binding.remote_pry in your code and then run pry-remote. This will attach your terminal to the pry debugging session.

binding.remote_pry # In your code
pry-remote # In your shell

• For memory efficiency, pow will shut down applications that are not used for some time (default is 15 minutes). Your application will take a few extra seconds to load the first time you come back to it.

Parsing null terminated key/value ASCII strings in Ruby

I’ve recently been messing with a lot of hex thanks to my current client. I ran across a key/value hex string where each key/value pair was NULL terminated (pairs separated from other pairs with a NULL character) and each key was seperated from its value by a colon. Here’s the hex string:

>> hexstr = "56494e3a3147314a433534343452373235323336370050524f544f3a3500504152414d533a302c312c322c342c372c392c31312c31342c323000494e44435452533a302830303030303030303030303131292c3128303131303031303131313129"

First I wanted to convert this to ASCII, Ruby makes this fairly simple:

READMORE

>> str = [hexstr].pack 'H*'
=> "VIN:1G1JC5444R7252367\x00PROTO:5\x00PARAMS:0,1,2,4,7,9,11,14,20\x00INDCTRS:0(0000000000011),1(01100101111)"

Now we have an ascii string with null terminators. Ruby has escape sequences like \n for newline and \t for tab. It also has hex escape sequences, for example the letter a is \x61 as 61 is its hex value. A null character is the hex value 00 so the escape sequence is \x00. Now to break the string apart we use split:

>> kv_pairs = str.split "\x00"
=> ["VIN:1G1JC5444R7252367", "PROTO:5", "PARAMS:0,1,2,4,7,9,11,14,20", "INDCTRS:0(0000000000011),1(01100101111)"]

Finally, you can split these pairs into further pairs by mapping over them with split and then use Ruby’s Hash[] syntax to create a hash from the array of arrays:

>> kv_pairs_array = kv_pairs.map { |key_vals| key_vals.split ':' }
=> [["VIN", "1G1JC5444R7252367"], ["PROTO", "5"], ["PARAMS", "0,1,2,4,7,9,11,14,20"], ["INDCTRS", "0(0000000000011),1(01100101111)"]]
obdii_info = Hash[kv_pairs_array]
=> {"VIN"=>"1G1JC5444R7252367", "PROTO"=>"5", "PARAMS"=>"0,1,2,4,7,9,11,14,20", "INDCTRS"=>"0(0000000000011),1(01100101111)"}

If you’re like me and don’t like upcased strings for keys, you can symbolize the keys (thanks active support) with this oneliner (I added the downcase call to the mix):

>> obdii_info = Hash[obdii_info.map { |(k, v)| [ k.downcase.to_sym, v ] }]
=> {:vin=>"1G1JC5444R7252367", :proto=>"5", :params=>"0,1,2,4,7,9,11,14,20", :indctrs=>"0(0000000000011),1(01100101111)"}

Looking nice! I’ve made these into methods for a util class I include. Eventually I’d like to make a special hex string class to handle this better, but for my current use case this works well.

Here are some methods you can throw in your own class:

def hex_to_ascii hex_string
  [hex_string.delete ' '].pack 'H*'
end

def hashify_null_term_str str
  Hash[str.split("\x00").map { |key_vals| key_vals.split ':' }]
end

def symbolize_keys hash
  Hash[hash.map { |(k, v)| [ k.downcase.to_sym, v ] }]
end

# Example usage:
symbolize_keys hashify_null_term_str hex_to_ascii "56 49 4e 3a 00 50 52 4f 54 4f 3a 30 00 50 41 52 41 4d 53 3a"
# => { vin: nil, proto: "0", params: nil }

Happy Hacking.