Sunday, September 7, 2014

Implementing session timeout in playframework

According to play documentation "There is no technical timeout for the Session. It expires when the user closes the web browser. If you need a functional timeout for a specific application, just store a timestamp into the user Session and use it however your application needs (e.g. for a maximum session duration, maximum inactivity duration, etc.)."

So I used the following way to implement a session timeout. Following custom authenticator class was used to implement this.

  public class ValidateUserSessionAction extends Security.Authenticator{

    @Override
    public String getUsername(Http.Context ctx) {
        long currentTime=System.currentTimeMillis();
        long timeOut=Long.parseLong(Play.application().configuration().getString("sessionTimeout")) * 1000 * 60;
        String temp=ctx.session().get(Constants.LAST_SEEN_KEY);
        if (temp == null) {
            temp = String.valueOf(currentTime);
        }
        if((currentTime-Long.parseLong(temp))<timeOut) {
            //If multiple instances are running, time should be synchronized between nodes
            ctx.session().put(Constants.LAST_SEEN_KEY, String.valueOf(System.currentTimeMillis()));
            return ctx.session().get(Constants.SESSION_USER_KEY);
        }else{
            ctx.session().clear();
            return null;
        }
    }

    @Override
    public Result onUnauthorized(Http.Context ctx) {
        return redirect(controllers.routes.UserController.signIn());
    }

    
}

Above authenticator class can be used to validate user actions like below.
 
 @Security.Authenticated(ValidateUserSessionAction.class)
    public static F.Promise<result< updateEmail() {
        //do something
        return ok();
    }

Saturday, September 6, 2014

Optimizing Apache Storm deployment

Recently we happen to run some pretty large Storm topologies in a Storm cluster which runs on Linux. When we running it there were two main issues occurred due to system limitations. First one was logged in Storm logs as,

“java.lang.OutOfMemoryError : unable to create new native Thread”.


We fixed this problem by increasing the Ulimit for Storm. Usually storm spawns processes with an user named Storm. So we have to increase the Ulimit for storm user. You can see the ulimits using command "ulimit -u". To increase the ulimits you can follow an approach like this.

The second problem was communication link failures between Storm nodes as well as communication link failures between other services (e.g. external APIs, databases, etc). To resolve this problem we had to enable tcp time wait reuse and also we increased the port range for tcp. hat can be done in following manner.

 Put these to /etc/sysctl.conf file and issue 'sysctl -f'
    net.ipv4.tcp_tw_reuse = 1
    net.ipv4.ip_local_port_range = 18000    65000

Note that we have to apply this setting for every node in Storm cluster. In addition to there were some errors like below due to Netty timeouts (Storm uses Netty as underlying messaging layer).

java.lang.RuntimeException: java.lang.RuntimeException: Client is being closed, and does not take requests any more
at backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:107) ~[storm-core-0.9.1.2.1.4.0-632.jar:0.9.1.2.1.4.0-632]
at backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:78)

This can be corrected by increasing netty timeout values in configurations.