Improving on the original AMQP chat application

Continuing from where my earlier post on AMQP left off, this post describes the changes introduced to make the application more object oriented, shifting from the fanout exchange to direct exchanges and new problems on long polling.

My initial example of the amqp_server would make most Ruby developers cringe. So a quick glance at the AMQP documentation (which is awesome) revealed a simple approach on how to structure the code. The complete source is available here

Here is the revised amqp_server

require 'rubygems'
require 'amqp'
require 'mongo'
require 'em-websocket'
require 'json'
require 'evma_httpserver'
require File.expand_path('../message_parser.rb', __FILE__)
require File.expand_path('../producer.rb', __FILE__)
require File.expand_path('../worker.rb', __FILE__)
require File.expand_path('../consumer.rb', __FILE__)
require File.expand_path('../http_server.rb', __FILE__)
require File.expand_path('../setup.rb', __FILE__)
require File.expand_path('../socket_manager', __FILE__)


# start the run loop
EventMachine.run do
  connection = AMQP.connect(:host => '127.0.0.1', :port => 5672)
  channel = AMQP::Channel.new(connection)

  socket_manager = SocketManager.new
  EventMachine.start_server('127.0.0.1', 8082, Setup, [socket_manager, channel])
# EventMachine.start_server('127.0.0.1', 8081, HttpServer, channel)

  puts "---- Server started on 8081 -----"


   EventMachine::WebSocket.start(:host => '127.0.0.1', :port => 9000) do |ws|

    ws.onopen do
      puts "EStaiblished......"
      ws.send('Connection open')

     puts ">>>>>>>>#{ws.request["query"]} <<<<<<<<< message[:roomname])
    end

    ws.onclose do
      puts " socket connection closed."
      roomname = ws.request["query"]["roomname"]
      username = ws.request["query"]["username"]
      SocketManager.new().remove_socket(roomname, ws)
    end
  end
end

Switching from Fanout to Direct Exchanges

The most significant change is switching from fanout exchanges to direct exchanges. This identifies the room to have a unique queue and is bound to an exchange with a routing_key = room_name. All new messages that arrive are published via the exchange with the same routing key. This works like a charm when working with websockets but won’t be a wise approach when using long polling or simple polling.

Why this works with sockets but is not the best design for polling

In this post we consider each room to have a queue and all messages directed to this room and available from this queue. With websockets its easy to keep track of users connected to this room and on arrival of a message its easy to broadcast this to all the websockets connected on that room.

Keeping track of websockets for a room. socket_manager.rb

class SocketAPI
  def self.api
    @sockets ||= {}
  end
end

class SocketManager
  attr_accessor :sockets

  def initialize
    @sockets = SocketAPI.api
  end

  def add_socket(roomname, sock)
    puts "=#{roomname}" * 50
    puts  "IN ADD SOCKET"
    puts "=" * 50
    puts "SOCKETS #{SocketAPI.api.inspect}"
    @sockets = SocketAPI.api
    if @sockets["#{roomname}"]
      puts "=" * 50
      puts "SOCKET HASH Exists"
      puts "=" * 50
      socket_array = @sockets["#{roomname}"]
      socket_array.push(sock)
    else
      #puts "=#{roomname.blank?}" * 50
      puts "SOCKET HASH DOES NOT Exists"
      puts "=" * 50
      @sockets[roomname.to_s] = []
      socket_array = @sockets["#{roomname}"]
      socket_array.push(sock)
    end
  end

  def remove_socket(roomname, sock)
    sockets = SocketAPI.api
    sockets["#{roomname}"].delete sock
  end
end

So all that is needed is identify the associated sockets for a room and push messages from the consumer to the browser. With polling however we would need to use the ‘pull api’ for queues. Heres an example


require "rubygems"
require "amqp"

EventMachine.run do
  connection = AMQP.connect(:host => '127.0.0.1')
  puts "Connected to AMQP broker. Running #{AMQP::VERSION} version of the gem..."

  channel  = AMQP::Channel.new(connection)
  queue    = channel.queue("c", :auto_delete => true)
  exchange = channel.direct("cexchange")

#  queue.subscribe do |payload|
#    puts "Received a message: #{payload}. Disconnecting..."
#    connection.close { EventMachine.stop }
#  end

  queue.bind(exchange, :routing_key => "cratos")


  exchange.publish "Hello, world!", :routing_key => "cratos"
  exchange.publish "Goodbye world", :routing_key => "cratos"

  exchange.publish "Goodbye world", :routing_key => queue.name


  q = channel.queue("c", :auto_delete => true)


  q.status do |message_count, consumer_count|
    messages = message_count
    consumers = consumer_count

    if messages > 0
      0.upto(messages - 1) do
        q.pop { |m, p| puts "#{m} Payload #{p}" }
      end
    end
  end
end

So we pop off messages from the queue one by one, but in this case we have no information about all the logged in users. So the messages are wiped out from the queue by the first poll that arrives from the members of the room. What this means is that the queue is wiped clean with the first poll updating one user’s browser window with the latest messages while the others see the old messages.

What are the alternatives

Right now it seems like “topic exchanges” would work. Every user could have a queue for each room. So the routing key could be something like “sid.harry_potter” where the period separates the username and the roomname. Maybe this is not the best alternative but the first one I could think of.

Ugly hack

One part of the code which seems like a horribly ugly hack to me is the Setup class which creates our queues. Setup.rb simply defines a basic EventMachine Server which listens on port 8082 for incoming requests. This request arrives when the user creates a new room (in our rails app).

The incoming requests triggers a call to the worker.rb which creates the queues. The other change aren’t that significant other than moving chunks to respective classes.

I still continue to use Websockets but using websockets for production wouldn’t be wise considering its limited adoption. Hence I dabbled with SocketIO and simple XHR-polling.

Though my current link to the source does not contain my experiments with Long Polling I would like to briefly go into the problems I faced.

Some of the basic issues would be violating the same origin policy and the easiest solution that I could come up with was to use Apache proxying.



   ServerName localhost
   DocumentRoot /etc/apache2/www/trackertalk/public/    
   ErrorLog "/private/var/log/apache2/dummy-host.example.com-error_log"
   CustomLog "/private/var/log/apache2/dummy-host.example.com-access_log" common
   ProxyPass /poll http://a.localhost:8081
   ProxyPassReverse	/poll http://a.localhost:8081
   RailsEnv development	



   ServerName a.localhost
   ProxyRequests Off
   ProxyPass / http://localhost:8081
   ProxyPassReverse / http://localhost:8081


All that is needed is to forward all requests that arrive to localhost/poll to a.localhost:8081 where our http_server would live. The http_server is another http_server that uses EM::Http_Server on port 8082 to handle incoming requests. It would need to be able to handle incoming requests (like a poll to check for messages in the relevant queue and forward it back to the client or a new message and add it to the correct queue.) and provide
the required reponse.

If you know or have built applications using AMQP and Ruby, I would love to know more about how production ready applications are structured and built. Any suggestions, corrections or feedback on my code would be awesome

Advertisements

RabbitMQ and Rails. Why my chat application failed.

Over the last few weeks I’ve been playing with RabbitMQ and Rails have been completely blown away by how awesome it is.

I started out to build a simple chat application which would allow users to create a room and log into the room and post messages to multiple users who were logged in. Though, this is probably not the best use case for using AMQP but it helped me understand how AMQP worked and maybe I’ll get to use somewhere more apt.

RabbitMQ

What is RabbitMQ
RabbitMQ is an open source, highly scalable messaging system which provides support for STOMP, SMTP and HTTP messaging. [www.rabbitmq.com]

It’s built using the Advanced Message Queuing protocol (AMQP) which provides APIs in Ruby and several other programming languages.

Installation

The easiest way to install RabbitMQ on the Mac is using homebrew. The Macports installation has many issues and I would recommend going with HomeBrew. It also takes care of all dependencies like the correct Erlang installation etc.

You can test your installation by moving into the installation ‘bin’ directory and running rabbitmq-server

If you end up seeing something like

 ...
 starting error log relay                                              ...done
 starting networking                                                   ...done
 starting direct_client                                                ...done
 starting notify cluster nodes 
 broker running

your good to go.

Creating our AMQP server

In Ruby we could run our AMQP server using the ‘amqp gem’ along with EventMachine.

The AMQP server code.

require 'rubygems'
require 'amqp'
require 'mongo'
require 'em-websocket'
require 'json'

....


@sockets = []
EventMachine.run do
  connection  = AMQP.connect(:host => '127.0.0.1')
  channel = AMQP::Channel.new(connection)
  puts "Connected to AMQP broker. #{AMQP::VERSION} "
  mongo = MongoManager.establish_connection("trackertalk_development")

  EventMachine::WebSocket.start(:host => '127.0.0.1', :port => 8080) do |ws|
    socket_detail = {:socket => ws}
    ws.onopen do
      @sockets < true)
        #AMQP::Consumer.new(channel, queue)

      elsif status[:status] == 'MESSAGE'
        full_message = " #{status[:username]} :  #{status[:message]}"
        exchange.publish(full_message)
      end
    end

    ws.onclose do
      @sockets.delete ws
    end
  end

end

The code snippet contains most of the amqp server code. The entire code lives inside the EM.run loop. Ignore the Mongo bit as it doesn’t play a part in our area of interest.

@sockets = []
EventMachine.run do
  connection  = AMQP.connect(:host => '127.0.0.1')
  channel = AMQP::Channel.new(connection)
...
 

In this snippet I created a new AMQP connection to localhost. When not provided with any other parameter the default port used by AMQP is 5672 with the username: guest and password :guest. Since the RabbitMQ installation will never (in most cases) have to be accessed from outside the server its safe to run the installation with the defaults.

The second line creates a new channel.

Once the channel is established the EM::Websocket block is used to allow modern browsers to connect to our server on port 8080

EventMachine::WebSocket.start(:host => '127.0.0.1', :port => 8080) do |ws|
    socket_detail = {:socket => ws}
    ws.onopen do
      @sockets < true)
        #AMQP::Consumer.new(channel, queue)

      elsif status[:status] == 'MESSAGE'
        full_message = " #{status[:username]} :  #{status[:message]}"
        exchange.publish(full_message)
      end
    end

    ws.onclose do
      @sockets.delete ws
    end

This is where the core logic lies. On the socket connect block I store the current socket in an array to use it later.

The ‘onmessage’ block is called when a new message arrives from the browser. Since I needed to dynamically create new exchanges and queues I used a very amateurish system. The message sent from the browser arrives with the following parameters in the JSON format

The following snippet is not a part of AMQP but a feature borrowed from XMPP to get my application working

{status:'status', username:'someusername', roomname:'someroomname', message:'somemessage'}

Status could have values from [‘status’, ‘message’, ‘presence’]
Username contains the ‘current_user’ name
Roomname indicates the ‘room’ the user has been logged into.

Status messages indicate that some kind of chore needs to be performed like subscribing to queues, binding exchanges etc.

Messages indicate that the message needs to be pushed to an exchange. When a message is published to an exchange, subscribing queues push the message to the browser using Websockets.

Presence messages indicate the users presence.

Why the entire application was a failure

The most important aspect while using AMQP is identifying the type of exchanges needed. In this case my assumptions on identifying rooms to be exchanges and users to have individual queues proved to be the reason for failure.

The reason why this assumption would fail is if a user queue would bind to multiple exchanges with each user having a unique queue assigned to him, a simple fanout exchange would push messages from different rooms to the user’s logged in chat room which is not what we want. This way users would end up receiving messages that are not relevant.

A ideal approach would have multiple users subscribe to a room (assumed to be the queue in this case) via a topic exchange or direct exchange.

I also believe that it would be logical to dynamically create queues and exchanges using the EM.periodic timer to check a NoSQL database or key-value store for new requests rather than using Websockets.

The AMQP server code is available here

window.unload, synchronous ajax and page loads

I’ve been working on a project that had me work with javascript based timers and required me to ensure certain data was updated before the user left the page. The obvious solution in this case was to handle the data updating on the window.unload event.

Now the task at hand was to ensure that on unload the quiz status (the object i was working with) needed to be updated to ‘DONE’ or something to that effect and the next object that was in the ‘NEW’ state needed to be fetched.

Synchronous AJAX (or whatever synchronous XMLHttpRequests are called)
The problem with using asynchronous ajax in such a situation would be that one cannot guarantee if the page refresh or a new request would wait long enough to see through the ajax request. Chances are the page may unload much before you receive confirmation that the request was completed. The advantage with synchronous ajax is that the page waits in a frozen state awaiting a confirmation before it loads the next request. Sync ajax in jquery

In most scenarios you would not need to have that confirmation but heres one valuable detail that I overlooked and may come in handy:

When you wait on a response the next request (which could have been caused by clicking a link or a refresh) to the server has already loaded and is waiting to be rendered while you wait on the unload event to display the confirmation of your cleanup action. In my case this was updating the state of the quiz object from new to done.

What I failed to acknowledge was that the new request is processed before the server actually receives the message from your ajax call. So if your next request depends upon the clean up action performed by your ajax call, the data that is waiting to be rendered is already stale (as it was fetched by working on data before the ajax updation took place)

Just to drive the point home, here are the sequence of events
1. Click Link/Refresh
2. Request for refresh or click fired
3. window.unload event fired (Response from the earlier request waiting to be rendered)
4. ajax call to update data on the server (Response from the earlier request waiting to be rendered)
5. ajax callback received
6. Data for click link/refresh rendered.

* the sequence is purely to illustrate the point.

A optimal solution would be to use filters if your using Rails.

This may sound trivial for most experienced developers but in case you overlooked this I hope this post saves you some time.

EventMachine: Deferring Time consuming tasks to a separate thread

A small update on my last post which focuses on building a sample twitter feed application using eventmachine and the twitter gem. If you’ve already gone through the documentation and references, especially the introduction to eventmachine PDF, you would have noticed that the reactor is single threaded and the EM methods aren’t thread safe. It also describes a simple example on how to use EM.defer to run time consuming tasks on a separate thread as a background process by using a thread from the thread pool.

Though there isn’t significant benefit in running our twitter feed application’s ‘update fetch’ operation as a background process but for purely illustrative purposes lets make our server perform updates on a separate thread. EM.defer provides a callback option that allows us to fetch values returned by the background thread and return it to the user in the main thread.

Update your event_machine_server.rb file with the following changes and run both the client and server.

The code is also available here

 
require 'rubygems'
require 'eventmachine'
require 'socket'
require 'json'
require 'hashie'
require File.expand_path("twitter_lib.rb", __FILE__ + "/..")
require File.expand_path("tweet_Fetcher.rb", __FILE__ + "/..")

class Echo < EM::Connection

 attr_reader :data, :response, :status, :fetcher   
 
 def post_init   
   @status = :inactive 
    ip, port = Socket.unpack_sockaddr_in( get_peername) #peer.methods.inspect
    puts  "Connection Established from #{ip} #{port}"
  end

  def receive_data(data)
    puts "[LOGGING: RECEIVED] #{data}"
    @data = JSON.parse(data)
    puts "[LOGGING: PARSED DATA ] #{@data} #{@data.class.to_s}"
    initialize_fetcher
    execute_request
    
  end

  def unbind
    puts "Connection Lost" + self.inspect
  end
  
  def respond
    send_data(@response)
  end
  
  def execute_request
    if @data["op"] == "fetch"
      puts "Please wait while we fetch the data ..."
      @status = :active
      response = @fetcher.fetch      
      send_data(response.to_json)
      Echo.activate_periodic_timer(self)    
    elsif @data["op"] == "update"
      puts "Fetching update . . ."
      response = @fetcher.fetch_update
    #  send_data(response.to_json)      
    end
  end
  
  def self.activate_event_machine(this = nil)
    EM.run do 
        puts "Starting echo server . . . ."
        EM.start_server('0.0.0.0', 6789, Echo)
        puts "STARTED "
    end    
  end
  
  def self.activate_periodic_timer(this = nil)
    response = nil
    operations = Proc.new do     
      this.update_operation
      response = this.execute_request
    end
    
    callback = Proc.new{ this.send_data(response.to_json)}
    
    EM.add_periodic_timer(2) do
      EM.defer(operations, callback)      
    end    
  end
  
  def update_operation
    @data["op"] = "update"
  end
  
  private  

  def initialize_fetcher
    @fetcher =  Fetcher.new({ :consumer_key => "xxxxxxxxxxxxxxxx",
                     :consumer_secret => "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
                     :oauth_token => "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
                     :oauth_token_secret => "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" 
                    })
                    
   # puts "[LOGGING FETCHER INITIALIZED] #{@fetcher.inspect}"                
  end
  
end

Echo.activate_event_machine

=begin
  EM.run do 
      puts "Starting echo server . . . ."
      EM.start_server('0.0.0.0', 6789, Echo)
      puts "STARTED "
    
      EM.add_periodic_timer(5) do
        puts "Timer activated"
      end    
  end
=end




Event Machine: A sample application to learn about event machine

I have been playing around with event machine for a couple of days now and have been trying to work out a basic (quick and ugly) example to understand how it could be used for simple applications.

Though my understanding of the subject is still very superficial here is a sample application that you could try to get yourself going. There are a lot of tutorials available for this and I guess this would be a good starting point

To get a quick idea of the working app try changing the value of the ‘timeline’ key in the tweet_fetcher file to :public.

Further bulletins as events warrant.

Javacript: Binding, call and apply methods

Continuing my interest in Javascript this would be second in the series of Posts revisiting the basics of Javascript.

One of the many things that I have been ignorant about when it comes to Javascript is the existence of bindings. Like Ruby ( and other languages) Javascript uses bindings to allow functions to be called with reference to a specific object.

I like to think of this as something similar to instance_eval in Ruby or the binding method.

Instance_eval evaluates an object in the context of an object its called with so lets take a simple example in Ruby


class Cat
def speak
 "miaow"
end
end

class Dog
def speak
"Bark"
end
end

d = Dog.new
=> #
>> c = Cat.new
=> #
>> c.instance_eval{ speak }
=> "miaow"

All this describes is that the speak method is invoked with reference to the instance c (Cat Object)

Similarly the binding method

class Person
  def bind
   @var = "Person Object"
  binding
 end
end

>> p= Person.new
=> #
>> b = p.bind
=> #
>> eval("@var", b)
=> "Person Object"

“binding” which is a kernel method provides us with a binding or snapshot of the context at a particular place in the code where it retains the execution context such as the variable values (@var in this example) which can be used at a later point. With eval we evaluate the value of the @var variable with respect to the binding generated earlier.

Similary Javascript provides us with the apply and call methods. Based on Christophe Porteneuve’s example on the listapart blog here is a simple example to illustrate the use of apply
and call methods


function Song(band, album, song){
	this.band = band;
	this.album = album;
	this.song = song;
	
	this.info = function(other){
		console.log(this);
			alert( 'Did you know '  + other.band + " covered " + this.band + "\'s " + this.song ); 
		}
		
	
	
}



var korn = new Song('Korn','Another Brick in the Wall', 'Another Brick in the Wall' );
var pink = new Song('Pink Floyd', 'The Wall', 'Another Brick in the Wall');

pink.info(korn);

In this case we see a message that says “Did you know Korn covered Pink Floyd’s Another Brick in the Wall”. So thats cool.

The problem occurs when the reference is lost (binding loss: when you assign a method to a variable or pass it as an argument). So lets append this to our earlier example

function display_info(fn, args){
	
		fn(args);
	
}

display_info(pink.info, korn);

Now we see “Did you know Korn covered undefined’s undefined”. The logger statement in the info method will show you that “this” actually is pointing to the window object and not Object of type Song. This is because the minute the binding it lost “this” starts referring to the default binding that is the window object.

So we solve the problem using call or apply. Add this to the bottom of the display_info method

	fn.apply(pink, [korn]);
	fn.call(pink, korn);

Now we see it works again. All we do here with the call and apply method is specify explicitly that we would be invoking the fn i.e. pink.info with reference to the pink object with the korn object as an argument.

So whats the difference between call an apply then? Nothing except that call expects individual arguments to be passed while apply accepts an array

        fn.apply(BindingObject, ArrayArgument);
	fn.call(BindingObject, Argument1, Argument2...);

The inferences drawn in this post have been learned from Christophe Porteneuve’s post titled Getting Out of Binding Situations in JavaScript from the ListApart Blog. Its an awesome post and definitely a must read if your interested in Javascript.

MultiUser Chat using XMPP and Orbited (Using Ruby-on-Rails)

One of the things that I wanted to understand and build since I first learned to program was to build a chat client. Something that would allow people to communicate and I am extremely thankful to Rishav Rastogi for introducing me to XMPP.

I never really understood all the moving parts very clearly during my first interaction with the technologies but with some time on my hands now I decided to revisit the entire process of building a web chat client. While there are a few well documented resources that cover how to build a simple web chat client the information is mostly directed towards using XMPP and building a one-on-one chat.
Though the requirements for building a multiuser chat aren’t significantly different there are subtle differences that exist.

A brief introduction to XMPP, Ejabberd (our XMPP server) and Orbited

XMPP
Ejabberd is an XMPP server that I used to build my chat client with.

The Extensible Messaging and Presence Protocol (XMPP) is an open technology for real-time communication, which powers a wide range of applications including instant messaging, presence, multi-party chat, voice and video calls, collaboration, lightweight middleware, content syndication, and generalized routing of XML data.Xmpp.org

The technology was initially called Jabber and hence both Jabber and XMPP are used interchangeably on several posts. – Wikipedia

Ejabberd
Ejabberd is a Jabber/XMPP server built using Erlang and is open-source and we would be using Ejabberd in this example. Another popular alternative for Ejabberd is OpenfireEjabberd

Xmpp4r
Since this post would be using Rails we use the xmpp4r gem which is a wrapper over the standard XML that XMPP/Jabber/Ejabberd uses, thus allowing us to work with Ruby rather than generate XML. For those using Ruby 1.9.2 the gem installation may throw up some errors while installing the Rdoc so I’d recommend you either skip the Rdoc installation or ignore the error. The online documentation for Xmpp4r is pretty good and the gem comes with some useful examples that could help you get started.

Orbited
Orbited provides a pure JavaScript/HTML socket in the browser. It is a web router and firewall that allows you to integrate web applications with arbitrary back-end systems.Orbited

Why do we need Orbited?
With our existing arrangement (once we install Ejabberd and xmpp4r gem) we could get a basic messaging system ready. We could have users send messages and receive messages. The problem would be to receive those messages on the browser. There is no way we can display those messages without having to poll our server to fetch this information and we know polling could cause scalability issues. Orbited fills this void by acting as a web router that routes the incoming messages to the appropriate user’s browser using a technique called as long-polling. And long-polling is more scalable than polling.

Long-Polling
Comet is a broad term used for technologies like Long-Polling and streaming. While traditional polling requires periodic requests to be sent to the server and then return with the response, in long-polling a connection is established with the server which persists until a response is provided (or the request times out). Once the response is provided the connection is closed and a new one is step up waiting for the next response from the server. Similarly a new connection is set up on timeout. In Streaming the connection persists between the client and the server while the information is transferred.

According to HTTP 1.1 a browser is allowed to have only 2 connections to the server one of which is used here for real time communication, though I am not fully clear if this is exactly the way the connection is setup. Apparently IE 8 allows 6 connections per host so I shall look forward to any clarifications on this.

Orbited comes with support for technologies such as STOMP, IRC and XMPP so its a handy tool to get started with.

Installation

Installing Xmpp4r
This is the easiest part especially with Rails 3. The following is a snippet of my gemfile

source 'http://rubygems.org'

gem 'rails', '3.0.3'

# Bundle edge Rails instead:
# gem 'rails', :git => 'git://github.com/rails/rails.git'

gem 'mysql2'
gem 'xmpp4r'
gem 'authlogic'
gem 'rails3-generators'

group :development do
	gem 'rspec-rails', '2.3.0'
	gem 'mongrel', '1.2.0.pre2'
	gem 'cgi_multipart_eof_fix'
	gem 'fastthread'
end

group :test do
	gem 'rspec', '2.3.0'
	gem 'webrat', '0.7.1'
end




bundle install and your ready.

Installing Ejabberd

You can download the installer from here. At the time of this tutorial the lastest version was 2.1.6.

The installer guides you on how to setup the xmpp server. Here are some of the questions you would have to provide answers to
Domain: siddharth-ravichandrans-macbook-pro.local This is simply a name (domain name) that you would want your server to be known by. In production this could be chat.example.com or jabber.example.com. For development the default is good. Its important that you note down the domain name somewhere as you will be using this a lot.

Cluster Installation: NO

Admin: siddharth This could be any name that you choose. This provides a way to access the ejabberd web administration interface
Admin password : siddharth

Thats it, you have your ejabberd server installed. Now open the folder you installed it in and navigate to the bin folder.

 ./ejabberdctl start 
 SIDDHARTH-RAVICHANDRANs-MacBook-Pro:bin SIDDHARTH$ ./ejabberdctl status
The node ejabberd@localhost is started with status: started
ejabberd 2.1.6 is running in that node 

The will let you know if the server is working

Now that we have confirmation that our server is running log onto http://localhost:5280/admin to access you admin interface.

You may log in as [AdminUser]@[domain] followed by the password.
In my case ‘Siddharth@siddharth-ravichandrans-macbook-pro.local’ with the password ‘Siddharth’

You should now be able to see a web console for the administrator

Installing Orbited

The version of Orbited that I used was 0.7.10 which is available here. Ensure that you have python 2.5 or higher installed in your system. Most linux and OS X systems come with Python pre-installed. You can check by

$ python 
Python 2.6.1 (r261:67515, Jun 24 2010, 21:47:49) 
[GCC 4.2.1 (Apple Inc. build 5646)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> exit()

Twisted is installed as dependency for Orbited 0.7.10 so it need not be installed explicitly but incase you face some errors these are the steps of installation

Ensure that the orbited.cfg file is placed in the /etc folder which is where orbited automatically checks for the configuration file or else it may be supplied as an argument

 sudo orbited --config=/Users/SIDDHARTH/orbited.cfg

Once your done open the configuration file on your favorite editor

[global]
reactor=select
#reactor=kqueue
# reactor=epoll
session.ping_interval = 40
session.ping_timeout = 30
# once the sockets are open, orbited will drop its privileges to this user.
user=SIDDHARTH

For the reactor epoll would the one to select on Linux machines and Kqueue for OS X but I noticed that Kqueue has not been maintained and throws errors so using select is the last resort. Though select has scalability issues its okay to use it for development.

Set the user to the user that you would want orbited to run as.
The access section identifies how orbited will communicate with Ejabberd
Orbited will listen to all incoming requests at port 8000 and communicate with port 5222 with XMPP (Ejabberd uses 5269 for server to server communication)
Therefore

localhost:8000 -> localhost:5222. 

In production this could look like

localhost:8000 -> example.com:5222

So our access section would look like

[access]
#localhost:8000 -> irc.freenode.net:6667
localhost:8000 -> localhost:5222
* -> localhost:4747
#* -> localhost:61613

Thats it, orbited is ready. Give it a go by typing ‘orbited’ in the console. You should see the server start.

Beginning with some XMPP programming
I will be working on some basics of using xmpp4r which are explained beautifully in François Lamontagne’s two part tutorial on using Jabber with xmpp4r

Once you’ve conquered the basics of user subscription and sending messages lets take a look at the Multi User Client support provided in Xmpp4r.

Registering our users to the Jabber server. Ideally this would be after a user registers to your site, so an after_create operation.

require 'xmpp4r'
require 'xmpp4r/muc'
require 'xmpp4r/roster'
require 'xmpp4r/client'
# getting done with all the requires so you can try this on the console
 client = Jabber::Client.new(Jabber::JID.new('first_user@siddharth-ravichandrans-macbook-pro.local'))
 client.connect
 client.register('password')

# do the same for another user with full_jiid = 'second_user@siddharth-ravichandrans-macbook-pro.local'

Logging into the server

require 'xmpp4r'
require 'xmpp4r/muc'
require 'xmpp4r/roster'
require 'xmpp4r/client'
# getting done with all the requires so you can try this on the console
 client = Jabber::Client.new(Jabber::JID.new('first_user@siddharth-ravichandrans-macbook-pro.local'))
 client.connect
 client.auth('password')

# Don't forget to log both users in

Logging into a room/ creating a room
The MUC Client is a multi User chat Client. The XMPP4R gem provides support for MUC too.

Create a new client

muc = Jabber::MUC::MUCClient.new(client)

The MUC is not to be confused with the room. Its simply a client that serves as an interface for the user in a particular room.

Joining/Creating a room

muc.join(Jabber::JID::new('chatroom@conference.siddharth-ravichandrans-macbook-pro.local' + client.jid.node))

This lets the user join a room called chatroom and the user is logged in to the room as client.jid.node which evaluates to first_user in our case.

The domain appends the word conference by default to all multi user chat rooms and can be changed by editing the configuration file. The JID for a room can be split as ROOM_NAME + @ + conference.domain_name/user_nick

Setting up callbacks for the client

    muc.add_join_callback do |m|
      puts "[NEW MEMBER JOINED] " + m.to.jid.node
    end

    muc.add_message_callback do |m|
      puts "[NEW MESSAGE]" + m.body
    end

    muc.add_leave_callback do |m|
      puts "[MEMBER LEFT] " + m.to.jid.node
    end

The callbacks like the one described earlier in François Lamontagne’s two part tutorial get called when a new user joins the chat room, sends a message to the room or leaves the chat room. The MUC chat is actually very similar to the one – on – one chat example described in François Lamontagne’s example except that when a message is directed to the room it relays the message to all of the members in the room. So if you look at the xml you will notice that a message directed to the room is eventually directed to each user in the chatroom. The only difference is the send method which belongs to muc object takes care of the relaying or you may query the roster (I will come to this in a moment) to identify the members in a room and post a message to each member.

Sending a message to the room

muc.send(Jabber::Message.new('chatroom@conference.siddharth-ravichandrans-macbook-pro.local', 'Pink Floyd is the greatest band ever'))

The message type for a message sent to a chatroom is automatically set to the type :groupchat. (Jabber::Message is explained here. Lets have a look at the associated xml that is sent to each member

In order to view xml generated set the Jabber::debug to true

 Jabber::debug = true

The roster describes the subscriptions or the buddies on a one-on-one chat but in a chatroom the muc client has a roster that identifies the number of users in a chatroom along with their presence

muc.roster 

would yield something like this

The MUC roster is extremely useful and allows you to set callbacks too.

This pretty much wraps my example using Ejabberd and XMPP4R. The next part of my post will briefly describe how we can use orbited and have this information flow through the browser.

Starting Orbited

orbited

Would get orbited up and running if you placed the orbited.cfg in the /etc folder. Once orbited is running you can log onto http://localhost:8000/static where you would be able to the see the javascript files that Orbited provides you with. You will notice Orbited.js and a static folder. Jump into the static folder -> then protocols -> Xmpp -> to find the xmpp.js file. We will be working primarily with these two files.

So first lets make these two files available to our application by putting them in a layout file.

You will notice two partials at the bottom of my layout file called _tcpsocket and xmpp_client (both poorly named).

Before we begin try running this snippet obtained from Micheal Carter’s Sockets in the Browserarticle on CometDaily.com. Add this snippet to the _tcpsocket.html.erb partial that is included in the layout.

Load a view page (which includes the layout containing this parital). It could be any scaffold generated code block.

	$(document).ready(function(){

		var conn  = new Orbited.TCPSocket();
		conn.open('localhost', 5222);
		conn.onopen = function(){ alert('connection opened');
			//	conn.send('Hello World');
		}
		conn.onread  = function(data){ alert('RECIEVE DATA' + data ); }

               conn.onclose   = function(data){ alert('connection closed'); }

	});

This would be a helpful example to understand better what Orbited does. All it does is opens a tcp socket on localhost and connects to port 5222 . The onopen callback is called when the connection is opened and sends a piece of text which is read by the onread callback and the connection close callback is called.

Basically reading the data whenever something is sent by the server while waiting for it with an open socket connection is what we do.

Looking at our _tcpsocket partial

	
	 
		document.domain  = document.domain;
		Orbited.settings.port  = 8000;
		Orbited.settings.hostname   = 'localhost';
		TCPSocket  = Orbited.TCPSocket;		
	

You may ignore the document.domain = document.domain code for now. Here we include the Orbited.js code along with the Xmpp.js javascript files provided by Orbited. We also specify the port we would be listening to and the hostname.

The xmpp.js file is where all the magic (not really) happens.

The xmpp.js contains (yet another) javascript based interface to XMPP methods, thus allowing us to perform all the XMPP operations right from the browser. The existing xmpp.js file comes with partial support for MUC operations. A poorly and hastily hacked xmpp.js file to suit basic MUC operations is available on my github account.

CONNECT = [""];
REGISTER = ["","",""];
LOGIN = ["","","Orbited"];
ROSTER = [""];
MSG = ["",""];
PRESENCE = [""];
EXT_PRESENCE = [];
GROUPCHAT_MSG = ["",""];

....

XMPPClient = function() {
    var self = this;
    var host = null;
    var port = null;
    var conn = null;
    var user = null;
    var domain = null;
    var bare_jid = null;
    var full_jid = null;
    var success = null;
    var failure = null;
    var parser = new XMLReader();
    self.onPresence = function(ntype, from) {}
    self.onMessage = function(jid, username, text) {}
    self.onSocketConnect = function() {}
    self.onUnknownNode = function(node) {}
    self.sendSubscribed = function(jid, me_return) {
        self.send(construct(PRESENCE, [me_return, jid, "subscribed"]));
    }
    self.connect = function(h, p) {
        host = h;
        port = p;
        reconnect();
    }
    self.msg = function(to, content) {
        self.send(construct(MSG, [full_jid, to, content]));
    }
    self.unsubscribe = function(buddy) {
        self.send(construct(PRESENCE, [full_jid, buddy.slice(0, buddy.indexOf('/')), "unsubscribe"]));
    }
    self.subscribe = function(buddy) {
        self.send(construct(PRESENCE, [full_jid, buddy, "subscribe"]));
    }
    self.send = function(s) {.....

If you notice these methods end up generating the exact same XML code (converted to utf8) and sent to the ejabberd. So no real magic there.

Our goal is to now use this API to perform the same operations on the browser. Here is a basic MUC chat javascript. Add this to the _xmpp_client.js on the layout file. The snippet contains some missing text so use the code here



		
		console.log('XmppClient partial loaded');
		var hostname                               = 'localhost';
		var domain                                 = 'siddharth-ravichandrans-macbook-pro.local';
		var bare_jid                               =  ' //close quote 
		var password                               = ' //close single quote
		var chatroom_domain                        = 'conference.' + domain;
	//	var username                             = bare_jid + '@' + domain;

		console.log('xmpp client connect request posted');
		
		function loginSuccess(){
			alert('Login Successful');
		//	xmpp_client.set_presence('available');
			alert(typeof ROOM_NICK);
		   if(typeof ROOM_NICK != 'undefined'){
				 xmpp_client.join_room(ROOM_NICK, chatroom_domain, bare_jid, 'available', null);		 
				 console.log('JOIN ROOM Called');
			}	
		}


		
		function loginFailure(){
			console.log('Login Failed');
		}

		function serverConnectSuccess(){
			alert('Server Connection Success');
			$('.presence-status').html('('+ 'Server Connected'+')');
			xmpp_client.login(bare_jid, password, loginSuccess, loginFailure);
			
		}

		function serverConnectFailure(){
			alert('Server Connection Failed');
		}
		

		var xmpp_client                            = new XMPPClient;
		xmpp_client.connect('localhost', 5222);
		

		xmpp_client.onSocketConnect                = function(){
			$('.presence-status').html('('+ 'On Socket Connected'+')');
			xmpp_client.connectServer(domain, serverConnectSuccess, serverConnectFailure);
			console.log('After COnnect Server is called');	
			xmpp_client.login(bare_jid, password, loginSuccess, loginFailure);	

		}
		
		xmpp_client.onPresence = function(ntype, from) {
			var username  = bare_jid + '@' + domain + '/Orbited'; 
			if(from == username){
				if (ntype == null){
					$('.presence-status').html('(available)');
				}
				else{
					$('.presence-status').html('(' + ntype + ')');
				}
			}				
		}
		
		xmpp_client.onMessage = function(jid, username, text) {
			$('.conversation-box').append('
' + username.split("/")[1] + ' says : ' + text + '
'); alert('JID' + jid.to_s + ' Username ' + username + ' Text' + text); } $(document).ready(function(){ $('.send-message-button').click(function(){ alert('Incoming message'); var message = $('#message').val(); // xmpp_client.msg('007@conference.' + domain, message); xmpp_client.groupchat_msg(message, chatroom_domain); return false; }); });

Note that the xmpp.js file has been modified slightly from what Orbited provides us and the _xmpp_client.html.erb uses this modified api hence the method parameters may appear strange when compared with the original xmpp.js file.

The ROOM_NICK parameter is defined in the view using a content_for :js block and would be available inside a chat room.

I hope this is useful and please let me know of errors or misinformation in my article. In case you are interested in having a detailed write up on the installation of all the software, add a comment and I will send you the write up as soon as possible. I have tried my best to attribute most references to their original authors and sources but in case I have forgotten any I would be glad to update it anytime.

 

Final chat screenshot

Chaining methods using ‘tap’ method

The Ruby tap method is one the methods that makes you wonder about the thought that has gone behind writing the language. It belongs to Ruby 1.8.7 and above.

What does tap do
Tap allows you to create a chain of methods while working on the intermediate results and returning the object at the end.

Ruby Documentation

For example

[1,2,3,4].tap{|o| o << 100 } # [1,2,3,4,100]

I find this comes in really handy especially if your working with named_scopes. Though the technique may not be the best but I find it to be a better option than using eval on user input. So here is the situation.

We have the user enter certain information on the kind of users he wishes to see on the screen. Ex: Customers, Adminstrators, Managers

He could also specify if the user is active or inactive (indicating whether he has full access to his account or not).

So based on his input we generate a query string for method chaining. Assuming that I have a named_scope for each situation in my user model

result_set.tap{|o| (o.blank?) ? o << "experts" : o << ".experts" if params[:type] == "Expert"}.
        tap{|o| (o.blank?) ? o << "customers" : o << ".customers" if params[:type] == "Customer"}.
        tap{|o| (o.blank?) ? o << "adminstrators" : o << ".adminstrators" if params[:type] == "Adminstrator"}.
        tap{|o| (o.blank?) ? o << "activated" : o << ".activated" if params[:activated] == 'true'}.
        tap{|o| (o.blank?) ? o << "deactivated" : o << ".deactivated" if params[:activated] == 'false'}
    end

   User.instance_eval { eval(result_set) }

How cool is that. 🙂

Customizing User login with Authlogic: find_by_login_method

This post would be redundant especially when Devise is the “in” thing but I love the minimality of Authlogic and I’m sure over time I’ll grow to like Devise but for now Authlogic does everything I want.

In a scenario where you want Authlogic to find Users from a subset of users while adhering to a specific condition that you set (for instance: login users from a set of users who have active accounts), you would need to define some kind of condition or scope to find these users.

A simple configuration that Authlogic allows is

class UserSession < Authlogic::Session::Base
  find_by_login_method  :find_by_activated_user
end

The find_by_login_method identifies the class method that would be called upon the User model to find the user.

In user.rb

  def self.find_by_activated_user(email)
    self.find_by_email_and_activated(email, true)
  end

where activated is a boolean attribute that identifies if a user has been blocked from using the site.