What is ORM? How Sails.js attributes validation works? How to validate Array size and Uniqueness in Sails.js? If you have any one of those questions. Read on.

What’s ORM?

ORM (Object-Relational-Mapper) is datastore agnostic using adapters to communicate with your favorite data-store. In another word, instead of using raw SQL statements (which maybe different based on which underlying database you choose), ORM provides an abstraction layer on top of the database and you can easily query, create or update records with the same source code. And it will be mapped to the underlying database statements by the different adapters.


Ruby on Rails use Active Record and Sails.js use WaterLine as the ORM for their framework. There are some similarity between Active Record and Waterline. Like model Query Methods, Validations and Associations. I have to say that in order to achieve the convenience that’s given from Active Record, WaterLine still have some way to go, but it is already doing a great job. That’s why I am using it for my production application.


How Sails.js Attributes Validation Works?

While doing my backend API service, I have the experience of finding a way to validate Array. By sharing my experience, you will understand how Sails.js attributes validation works. I am using MongoDB. The attribute data type supported contains Array. And I have a need to validate the Array size and the uniqueness among the elements. Initially the maxLength validator strikes me, because array also support the length method, so this maxLength validator should support both String and Array. However, I was wrong. I did some digging and here is why it is not working. You should do so as it is good for  you to understand how the framework works.

Look at WaterLine package.json file:

"dependencies": {
    "lodash": "~2.4.1",
    "async": "~0.9.0",
    "anchor": "~0.10.0",
    "q": "~1.0.1",
    "waterline-schema": "0.1.11",
    "node-switchback": "~0.1.0",
    "waterline-criteria": "~0.10.7",
    "prompt": "~0.2.12",
    "deep-diff": "~0.1.6"
  },

Waterline depends on ‘Anchor’, which is a javascript library that lets you define strict types and validates attribute values. And the validation from ‘Anchor’ is actually done by package ‘Validator’. Again, you can check this from the package.json under ‘Anchor’ package.


Go ahead and check rules.js file in ‘Anchor’. You can find all the validations rules documented in Waterline documents here. The “maxLength” validator method is defined as below:


var validator = require('validator');
...
'maxLength'	: function (x, max) { return validator.isLength(x, 0, max); }

How to Validate Array Size and Uniqueness in Sails.js?

Based on the code, we can tell that maxLength in the end calls validator.isLength method. So we need to go and check “Validator” package’s validator.js file.

validator.extend = function (name, fn) {
    validator[name] = function () {
       var args = Array.prototype.slice.call(arguments);
       args[0] = validator.toString(args[0]);
       return fn.apply(validator, args);
    };
};
//Right before exporting the validator object, pass each of the builtins

//through extend() so that their first argument is coerced to a string
validator.init = function () {
   for (var name in validator) {
      if (typeof validator[name] !== 'function' || name === 'toString' ||
         name === 'toDate' || name === 'extend' || name === 'init') {
            continue;
      }
      validator.extend(name, validator[name]);
   }
};
....
validator.isLength = function (str, min, max) {
    var surrogatePairs = str.match(/[\uD800-\uDBFF][\uDC00-\uDFFF]/g) || [];
    var len = str.length - surrogatePairs.length;
    return len >= min && (typeof max === 'undefined' || len <= max);
};
..
validator.init();

Looking at above source code. When the "validator" is required, it will call the init() method, which actually convert the first arguments into type of 'String', which is called again on the extend() method.


By drilling down the source code, you should have a better understanding why maxLength doesn't work for Array. So what's the solution? You have to use custom validators.


Below is the sample code used in my Instameet server backend. With this code, you should be able to get started with the custom validations. In fact, the code is very simple and Sails.js is quite flexible.


 module.exports = {
  maxAttendees: 3,

  /**
   * Custom types and validations
   */
  types: {

    // Limit the max attendees to the property maxAttendees
    limitedAttendees: function(attendees) {
      return attendees.length <= Meet.maxAttendees;
    },

    // Make sure that the attendees are within the list of applicants
    withinApplicants: function(attendees) {
      var lastAttendee = attendees.slice(-1)[0];
      return this.applicants && this.applicants.length > 0 && this.applicants.indexOf(lastAttendee) >= 0
    },

    // Make sure that the newly added Array element doesn't exists in the existing Array
    uniqElement: function(array) {
      var lastElement = array.slice(-1)[0];
      return array.indexOf(lastElement) < 0;
    }
   },
.....
    confirmedAttendees: {
      type: 'ARRAY',
      limitedAttendees: true,
      withinApplicants: true,
      uniqElement: true
    }
}

Sails.js

Choosing API framework

I have been using Sails.js and Mongo DB to develop the backend server to provide API service for our mobile app Instameet, which is an apps that allows strangers to treat one another for dinner or coffees. In another word, it is a dating apps but based on some casual events. Why I choose Sail.js instead of Rails is because of its scalability and ease of development for the API. Ruby on Rails was designed to build large scale website but Sails.js is more designed for API development.

 

Introduction to Sails.js

This is the first time I am using Sails.js, which is based on the popular Node.js framework Express.js. “It is designed to mimic the MVC pattern of frameworks like Ruby on Rails, but with support for the requirements of modern apps: data-driven APIs with scalable, service-oriented architecture.” – statements from Sails.js official site.

Why Sails.js?

Why I choose Sails.js instead of Express.js is partially because of my background as well. I am a Ruby on Rails developer for 5 years now. So the way Sails.js works is very familiar to me as it is designed to mimic Ruby on Rails. It has scaffolding to quickly get you started on creating API service for your mobile apps. The problem with Node.js for me is that there are just too many frameworks too choose from. If not careful, and the framework you chose stopped supporting and upgrading, then you will have a big problem. Although Sails.js is not yet V 1.0 now, I saw great potential in this framework and it could be the next Ruby on Rails thing in the developer community.

First Experience

I have been using Sails.js for a few weeks now. The experience so far is still good. With minimum codes, you can get things up and running very quickly.

I will open source the source code for the backend i have done in near future. So for those who wanna learn about Sail.js can be beneficial. Stay tuned…

I experienced a weird problem tonight after checking out the latest source code. After running all pending migrations, the app is complaining that there is no such column overall_rating in database. The field overall_rating exists in db/schema.rb. It turns out that one of the db migration file is missing in the repository. In order to trace out what happened, below is the git command that I used:

git log -S overall_rating

This will show a list of logs that involve any file changes that include overall_rating. This will drill down to quite a few commits. Then using git show commit-hash command, i am able to know the migration file name from the first commit. Using below command will list the hash for the last commit for the migration file.

git rev-list -n 1 HEAD -- db/migrate/2014xxxx_add_calculation_xxxx.rb

Then using command git show again, i can tell that one of my colleges mistakenly delete the migration file. To restore it, i can run command:

git checkout commit-hash-value -- db/migrate/2014xxxx_add_calculation_xxxx.rb

Memcached is a free and open source, high performance, distributed memory object caching system. It is an in-memory key-value storage system that can be used to scale your application as a cache storage. Refer to http://memcached.org/ It can be used easily with Ruby on Rails by configuring the cache store.

config.cache_store = :dalli_store, 'ip_of_memcached_server:11211', {namespace: 'InvestingNoteWebApp', compress: true, expires_in: 1.day, username: 'sasl_username', password: 'sasl_password'}

Dalli is a high performance memcached client for accessing memcached servers. To use it you need to add dalli to the Gemfile.

gem 'dalli'

The dalli client usage is actually simple. The complicated part is setting up the memcached server. I am using Ubuntu as the server. The complexity lies on setting up memcached with SASL support. “Most deployments of memcached today exist within trusted networks where clients may freely connect to any server and the servers don’t discriminate against them. There are cases, however, where memcached is deployed in untrusted networks or where administrators would like to exercise a bit more control over the clients that are connecting.” – https://code.google.com/p/memcached/wiki/SASLAuthProtocol The SASL authentication protocol is used to protect the memcached from unauthorised access.

1. Use package installer to install

apt-get install memcached

Output:

Reading package lists... Done
Building dependency tree
Reading state information... Done
Suggested packages:
  libcache-memcached-perl libmemcached
The following NEW packages will be installed:
  memcached
0 upgraded, 1 newly installed, 0 to remove and 189 not upgraded.
Need to get 74.1 kB of archives.
After this operation, 227 kB of additional disk space will be used.
Get:1 http://archive.ubuntu.com/ubuntu/ precise-updates/main memcached i386 1.4.13-0ubuntu2.1 [74.1 kB]
Fetched 74.1 kB in 1s (61.9 kB/s)
Selecting previously unselected package memcached.
(Reading database ... 41818 files and directories currently installed.)
Unpacking memcached (from .../memcached_1.4.13-0ubuntu2.1_i386.deb) ...
Processing triggers for ureadahead ...
ureadahead will be reprofiled on next reboot
Processing triggers for man-db ...
Setting up memcached (1.4.13-0ubuntu2.1) ...
Starting memcached: memcached.

After doing this memcached will be installed as a service, the script is located at /etc/init.d/memcached. It also created a configuration file at /etc/memcached.conf. The purpose of having the memcached installed from package is that it can auto generate the service script and can start/stop easily.

By now, if you modify the /etc/memcached.conf with option -S, which means start the memcached with SASL enabled. It will throw error if you check the log tail -f /var/log/memcached

root@prod1:~# tail -f /var/log/memcached.log
This server is not built with SASL support.

We need to recompile the memcached at next step so that it supports SASL.
2. Reinstall memcached from source

apt-get install libsasl2-2 sasl2-bin libsasl2-2 libsasl2-dev libsasl2-modules libevent-dev
wget http://memcached.googlecode.com/files/memcached-1.4.15.tar.gz
tar -zxvf memcached-1.4.15.tar.gz
cd memcached-1.4.15
./configure --enable-sasl
make
make install

After this, you are actually have two copies of memcached. One under /usr/bin/memcached (without sasl support) and one under /usr/local/bin/memcached (with sasl support).
3. Update the /etc/initit.d/memcached script
Under this script, you will find the line

DAEMON=/usr/bin/memcached

Replace it with

DAEMON=/usr/local/bin/memcached

So that when you start the service it is actually start the memcached with sasl support. Alternatively, you can also overwrite /usr/local/bin/memcached to /usr/bin/memcached so that it only has one same copy of memcached.
4. Setting username and password

saslpasswd2 -a memcached -c memcached_user

It will prompt you for the password. After doing this, it will store the username and password under /etc/sasldb2. You can verify by running below command:

root@prod1:~# sasldblistusers2 -f /etc/sasldb2

5. Further configure memcached

# Run the daemon as user memcached. Make sure you have created the user memcached with command useradd.
-u memcached

# Specify which IP address to listen on. The default is to listen on all IP addresses
# This parameter is one of the only security measures that memcached has, so make sure
# it's listening on a firewalled interface.
-l private_ip_here

# Enable the SASL
-S

Now you can start the service. However if you try to connect with dalli client, you may see the Authentication failed error.

Loading production environment (Rails 4.0.4)
2.0.0 :001 > Rails.cache.fetch 'foo'
Dalli::Server#connect 10.130.243.100:11233
Dalli/SASL authenticating as investingnote
DalliError: Error authenticating: 32
 => nil 

When this happened, you can start the server under verbose mode with option -vv to check the error messages. On my case, the error is like below:

mech:  ``PLAIN'' with 27 bytes of data
SASL (severity 1): unable to open Berkeley db /etc/sasldb2: Permission denied
SASL (severity 2): Password verification failed

The problem is you start the memcached server with user ‘memcached’ but the file sasldb2 is created by root. The solution is to chown for the sasldb2 file to the user you set under -u option when you run the memcached.

chown memcached:memcached /etc/sasldb2

After this, you should be able to start the service memcached nicely.

We have set up the rails application with nginx as a reverse proxy. However, the following line of code is giving some trouble:

%a{:href => root_url(email: invitation.email_address, token: invitation.token)}

The root_url is supposed to render the absolute url with domain names. However, it rendered in production as “http://backend/?email=a%40a.com&token=1″ instead of “http://investingnote.com/?email=a%40a.com&token=1″. In another word, the Rails root_url is using the wrong host name with reverse proxy set up.

The problem is caused by the following Nginx set up.

upstream backend {
  server prod2:8000;
  server prod1:8000;
}
server {
        # other config here...
        location @puma {
          proxy_pass http://backend;
        }
}

When the backend puma server received the request, it thinks that the host name is ‘backend’ instead of the actual host. The solution is to preserve the Host header when using proxy pass.

server {
        # other config here...
        location @puma {
          proxy_pass http://backend;
          proxy_set_header Host $host;
        }
}