Thus during its lifecycle a ticket has to go through several stages. Most of these stages are irrelevant to the reporter of the ticket. He doesnt want to be overwhelemed by email notifications for each and every change to the ticket. Our managers used to get complains from clients who were ticket reporters about these notifications. We needed a solution.

One option was to turn off the the reporter notification completely other option was to modify trac’s default behaviour. The first option was not a good one as the reporters needed some kind of notification when a ticket was created, closed or reopened.

I searched Trac-hacks plugin list to find out something suitable. I couldnot find anything therefore i decided to tweak the Trac myself. It was not at all difficult as Trac has a plugin architecture and I found an api which could solve my problem. I just needed to add and modify few lines.That was it our problem was solved.I named it “FlexibleReporterNotification” plugin.

I have posted the solution at trac-hacks site so that someone needing similar solution may find it useful. It works on Trac 0.11.

Plugin url :- http://trac-hacks.org/wiki/FlexibleReporterNotificationPlugin

OS:-Windows 2003

Email client: - Fetchmail

Email2trac version: - 0.30

Others: - cygwin

Through this blog I intend to share our experience of configuring email2trac plugin on windows 2003. We had to make few changes in the original email2trac.py file in order to get it working.

We configured this plugin one eventful night a couple of months back. A normal trac plugin takes around 30-40 minutes at max to configure, but this one plugin took one whole night (around 6-7 hrs.). We started off with downloading the plugin and following the install instructions. After completing the instruction set we tried to test the plugin, the test failed we could not create tickets from email. We revisited the instructions to figure out if we had missed out something. We had followed the instructions quite well.

After playing around with the configuration files and revisiting the entire setup it was time to look at the code itself (email2trac.py). It took us a couple of hours to understand the code and its flow. Few things in the code did not make sense to us therefore we commented out those added few lines of ours and email2trac was working for us. Following are the things we commented out from email2trac.py

1) Blacklisting section.

2) Attachment Encoding.

3) Check for syslog.

4) Normalization of the file name for the attachment.

Due to the commenting of the above mentioned sections attachment path was not being generated correctly. To overcome this problem we added the following lines at the end of attachments definition after fd.close (line number 1073.)

attachmentPath = self.PROJECTPATH + “/attachments/ticket/” + str(ticket['id']) + “/” + url_filename

shutil.copy2 (temppath, attachmentPath)

Although this fix made the email2trac functional on our trac system, we are still struggling with attachments. We are able to send in zipped files in but the MS office documents get clobbered. We are still working on this and hopefully will resolve it soon at that point I will post another blog here.

In my view if any one wants to implement email2trac on windows they should be willing to make their hands dirty. There is not enough documentation available for this plugin. It is still a UNIX based program which can be made to work on windows with some hacks. Just as in our case we broke away from the program flow to make it work for us.

Following links are helpful in setting up this plugin on windows.

https://subtrac.sara.nl/oss/email2trac/wiki/Email2tracInstallation/Windows

http://functionalelegant.blogspot.com/search/label/Trac

Restart the nginx server:

/etc/init.d/nginx restart

For one of my rails application I am using this domain mail account setup for sending mails.
But I am not able to send the mail with the default smtp setings we use for sending mail.

So I search on net and I found the Goggle Apps consider its in different way. Which is TLS ans SSH service, which is not by default comes with the Action mailler.

ActionMailer can’t send emails using GMail out of the box. To add this functionality do the following configuration in your rails application:

1. Create the file lib/smtp_tls.rb

   require "openssl"
   require "net/smtp"
   
   Net::SMTP.class_eval do
   private
   def do_start(helodomain, user, secret, authtype)
   raise IOError, 'SMTP session already started' if @started
   check_auth_args user, secret, authtype if user or secret
   
   sock = timeout(@open_timeout) { TCPSocket.open(@address, @port) }
   @socket = Net::InternetMessageIO.new(sock)
   @socket.read_timeout = 60 #@read_timeout
   
   check_response(critical { recv_response() })
   do_helo(helodomain)
   
   raise 'openssl library not installed' unless defined?(OpenSSL)
   starttls
   ssl = OpenSSL::SSL::SSLSocket.new(sock)
   ssl.sync_close = true
   ssl.connect
   @socket = Net::InternetMessageIO.new(ssl)
   @socket.read_timeout = 60 #@read_timeout
   do_helo(helodomain)
   
   authenticate user, secret, authtype if user
   @started = true
   ensure
   unless @started
   # authentication failed, cancel connection.
   @socket.close if not @started and @socket and not @socket.closed?
   @socket = nil
   end
   end
   
   def do_helo(helodomain)
   begin
   if @esmtp
   ehlo helodomain
   else
   helo helodomain
   end
   rescue Net::ProtocolError
   if @esmtp
   @esmtp = false
   @error_occured = false
   retry
   end
   raise
   end
   end
   
   def starttls
   getok('STARTTLS')
   end
   
   def quit
   begin
   getok('QUIT')
   rescue EOFError
   rescue OpenSSL::SSL::SSLError
   end
   end
   end

2. Add the following lines to config/environment.rb and replace the values with the appropriate username and password:

   require 'smtp_tls'
   ActionMailer::Base.perform_deliveries = true
   ActionMailer::Base.raise_delivery_errors = true
   ActionMailer::Base.server_settings = {
   :address => "smtp.gmail.com",
   :port => 587,
   :domain => "mydomain.com",
   :authentication => :plain,
   :user_name => "username@mydomain.com",
   :password => "password"
   }

   ^*Note:To work your domain with “mydomain.com” you have to configure your MX records to work Gmail mail sending functionality out of box.

   Else you can use a temporary address @mydomain.com.test-google-a.com, I have not tested with this as I have already configure my MX records.

   You can get more information on changing http://www.google.com/support/a/bin/answer.py?answer=33352.

   Hope this information is useful to you.

After looking around for next couple of hours, I came across Thickbox. Now how could I miss this at the first place. It was what I exactly wanted for sometime now. And it had support for iframe also. So I went ahead and downloaded the required files namely jquery-latest.js, thickbox.js and thickbox.css. For those who don’t know about jquery – Jquery is a Javascript library to handle browser events, add AJAX features to your application etc. In next 15 minutes I was ready with the sample code to check if things really work and to my disappointment it didn’t work and moreover my existing AJAX features broke. When I opened the jquery library I found that it also uses ‘$’ like prototype.js, hence a conflict.

The solution for this problem is described here.

After making the required changes it worked. I was happy, client was happy until one fine day SSL came into picture. I use nginx as my web server and had done SSL setup before. I could setup the SSL but when I clicked one of the links which use thickbox to open up the popup, it just opened the content of the popup on a new page. After playing with the code from sometime I found that the j-query file gets truncated because its too big and also the page takes lot of time to download.

Jquery provides a compact version of its library. Using this file solved my problem.

EC2 is a web service that helps in elastic computation across the web. It is elastic because the computation is very resizable. When compared with Plain Hosting it provides the following advantages:

• Unlike traditional hosting services EC2 provides resources that can be used unlimitedly,
• EC2 provides full control on the resource where in the entire resource configurations can be manipulated according to the users requirement and
• EC2’s service doesn’t require users to pay a fixed, up-front fee irrespective of their actual computing power used but instead the pay is very dynamic and the computed based on the usage of the resource.

The usage of the EC2 image becomes easier if ruby on rails application is to be hosted. As rails provide gems for additional utilities, it provides a gem called as “ec2onrails” that makes the hosting even simpler.

The following are the steps in using the EC2 image:

• Install the gem – gem install ec2onrails

• Add the configuration files to the rails application – There are 3 files that is essential in order to use the image:

  • Capfile – The Capistrano file that includes the line, require ‘ec2onrails/recipes’ in addition to the existing config details. This makes the ec2onrails libraries available during deployment.

  • S3.yml – Instance stores appear to an instance as a local disk. They just survive intentional and unintentional reboots of the instance unless the instance terminates or the underlying drive fails. Hence the backup or replication of important data is very essential and it happens in Amazon S3. This happens using the configurations in this S3.yml file. The details comprises of aws_access_key, aws_secret_access_key and bucket_base_name. aws_access_key and aws_secret_access_key are the keys received at the time of creating an account with Amazon EC2 and bucket_base_name is the name of the bucket into which the data needs to be stored.

  • deploy.rb – This is the deployment file containing the Capistrano configurations for ec2onrails.

• Start the instance of the image – The instance can be started as follows:

  • Signup for Amazon EC2 and S3 to acquire the necessary authentication credentials – The necessary authentication credentials are the privacy encoded mail security X509 certificate, privacy encoded mail security private key file and the account ID.

  • Installing the EC2 Tools – This basically involves no installation as such because all what is required is to get the tools downloaded from the EC2 site, unzip it into a folder. The unzipped files are the pre-installed files, hence to make the tools visible, an environment variable – EC2_HOME is set with the path to the EC2 tools folder. Apart from this we need to tell the tools for whom it is going to be used for by setting two more environment variables such as EC2_PRIVATE_KEY and EC2_CERT containing the paths of the private key file and X509 certificate.

  • Start running the instance – All what we are left now is with running the instance. In order to run the instance we need to do the following:

    • Find a suitable AMI(Amazon Machine Image) – This happens to be the crucial step. Why I say that this is a crucial one is because on executing the command: ec2-describe-images –o self –o amazon, it will list all the available images. We need to pick the most stable image that will suffice our purpose. This is where I initially went wrong. What I did was, I chose an image without reading the name of the image file, which happened to be a very unstable one and hence had to install all the required software manually which had dependencies issue and as a result of this I had to spend several additional hours in the setup process. But realized pretty early that I was in the wrong track and then chose the right one. So how do we choose the right image? In the listing we need to look for developer-image which is public and available and make a note of the ami ID(something like ami-xxxxxxxx).

    • Generating the key pair – The privacy enhanced mail security files is used for all the interactions with the image but each and every time we interact we need to use then in our command with various options. This can be avoided by generating key pair out of the two security files and use this key pair for all the usage. In order to generate the key pair execute the command: ec2-add-keypair <name>. The name can be anything and this list a key pair which needs to be stored in a file with the name of our choice.

    • Run the instance – On executing: “ec2-run-instance ami-xxxxxxxx key_pair_filename”, the instance starts running. But it takes long time to start. This is because Amazon EC2 moves the images around the network before they can be launched. For big images and/or congested networks, this takes several minutes. To improve performance, images can be cached. As you launch your images more frequently, it becomes less noticeable.

    • Authorize Network Access o the instance – The instance uses port 80 and 22 which can be authorized by executing: “ec2-authorize default -p 22” and “ec2-authorize default -p 80”. If your application requires nay other port for access, it can be done the same way.

• Fetch the Public key – The Capistrano deployment requires the public key of the server which can be fetched by executing: “cap ec2onrails:get_public_key_from_server”

• Finally deploy the application with Capistrano – Firstly setup the ec2onrails Capistrano deployment by saying, cap ec2onrails:setup and then the first deployment can be done by executing, cap deploy:cold and for the later successive deployments just use, cap deploy. Apart from this the Capistrano with ec2onrails provides cap ec2onrails:db:archive and cap ec2onrails:db:restore. They can be explicitly executed for archiving and restoring the data for the database. But on the cold deploy execution the ec2 instance starts the incremental archiving of the data using the predefined cron jobs.

We now have our application hosted on EC2. Yes, it is that easy.

The only question that one would have in his/her mind now is, what will happen if my instance or the ec2 server terminates? Once the server terminates, the instance is lost and can never be got back. This is one feature that ec2 lacks. But it provides a solution to handle such situation.

There is something called as bundling of the image. On bundling the image, the instance configuration and the application is uploaded to the S3 which can then be register to another instance and the application can be in form again. This can be achieved as follows:

Prepare for bundling – The server requires the two privacy enhanced mail security files. This can be got by using scp command.

Example - scp pk-xxyyzz.pem cert-xxyyzz.pem root@ec2-xx-xxx-xx-xxx.compute-1.amazonaws.com:/mnt -i keypair_file_name

Bundle the current image – From the ec2 instance bundle the image.

Example - ec2-bundle-vol -d /mnt -k /mnt/pk-xxyyzz.pem -c /mnt/cert-xxyyzz.pem -u account_number -r i386

Upload the AMI – The AMI is then uploaded to S3.

Example - ec2-upload-bundle -b bucket_name -m /mnt/image.manifest.xml -a access_key_id -s secret_access_key

Register the Uploaded image – The image needs to be register from the local system and can be done as: ec2-register bucket_name/image.manifest.xml

This gives an ami ID with which the instance can be run.

EC2 - a quick and cost effective way of hosting Web Applications.

If min and max are the 2 variables that are holding minimum and maximum values, & random is the random number that is to be generated, in java the random number is calculated in this way…

random = (Math.random() * (max - min) ) + min


Here Math.random() is a function in java that generates any random value between 0 & 1… We multiply that with the difference between max & min values and finally add the obtained result with the min value to get a random number between 2 given numbers.. This idea was set as the basement for our requirement too… Wondering how?? This is how it goes…

We use Calendar object which defaults to current date and time. We then set Hour, minute & second in the same object(If required we can set the year, month & date too).. Now we have the Calendar object with the date what we set or current date and time. Calendar class has a method named getTimeInMillis which will convert this timeStamp to a long value. This long value serves as the first min variable as in the example where we generate a random number between 2 different numbers.

Now in the same way we get the long value of the second timestamp too which is the max variable value. Using a similar calculation as above we get a random long number between 2 different long numbers which are the long time. Finally we construct the date object with the long value that we obtained.. This date object’s value is the random time stamp between 2 different Calendar objects and hence the we can generate random date/time between two different Calendar objects!!!!

Here goes the code for that..

//code to generate random timestamp between morning 6 to evening 8 pm
public static void main(String args[]){
Calendar cdr = Calendar.getInstance();
cdr.set(Calendar.HOUR_OF_DAY, 6);
cdr.set(Calendar.MINUTE, 0);
cdr.set(Calendar.SECOND, 0);
long val1=cdr.getTimeInMillis();



Random r=new Random();
long randomTS=(long)(r.nextDouble()*(val2-val1))+val1;
Date d=new Date(randomTS);
System.out.println(d.toString());
}

PS: This will work even for generating random date between two different dates which can be a month/year gap.

One of the main reasons to move to Hibernate is to improve performance of the entire application. Traditional JDBC application is expensive and a complex job for any developer (if no DBA is available) to write the SQL queries. The developer first has to get a connection to the Database, create a Statement/PreparedStatement, ResultSets and iterate the ResultSet and embed the values into the ValueObjects (VO). Once done, the developer must ensure to close all the resources that were open during this process.

How does Hibernate eliminate all these? Hibernate requires only a DataSource to be specified and all the work of opening a connection, creating statements and executing these queries are the dirty job done by Hibernate. This shall be point of discussion in the later posts.

Let’s take a complex SQL query and convert them to Hibernate and see its advantages.

SQL Query:

select modality_type,
count(case when scheduled_ts >= DATE(NOW()) and scheduled_ts < DATE(NOW() + INTERVAL 1 DAY )
and hospital_id= 'BS' then scheduled_ts else NULL END ) as scheduled,
count(case when completed_ts != '0000-00-00 00:00:00'
and scheduled_ts >= DATE(NOW()) and scheduled_ts < DATE(NOW() + INTERVAL 1 DAY )
and hospital_id= 'BS' then completed_ts else NULL END) as completed
from order_status where modality_type in ('CT','DX','NM','US','IR','MR','PX','VA')
group by modality_type


This query is a typical Cross-tab/Pivot query. The query results in showing three columns - Modality Type, Count of Exams Scheduled for today and Count of Exams completed for the day which is grouped by Modality Type.

Let’s see how this is done with Hibernate.

1. Create a value object/java bean
Lets call our VO as CompletedAgainstScheduled

Note: For easier understanding I’ll show only a few snippets


/**
* @hibernate.class table="order_status" lazy="false"
*/
public class CompletedAgainstScheduled implements Serializable {
private Long id;
private String hospitalId;
private String modalityType;
private String completedTimestamp;
private String scheduledTimestamp;
private int completed;
private int scheduled;
private int interpretedBy;



/**
* @return
*
* @hibernate.meta attribute="id" value="ID"
* @hibernate.id type="long" column="id" generator-class="native"
*/
public Long getId() {
return id;
}
public void setId(Long id) {
this.id = id;
}

/** other setter and getters with hibernate mapping */

…..

/**
* get and set completed orders
*
* @return
*
* @hibernate.meta attribute="completedTimestamp" value="Completed Timestamp"
* @hibernate.property column="completed_ts" formula="case when completed_ts != '0000-00-00 00:00:00' then completed_ts else null end"
*/
public String getCompletedTimestamp() {
return completedTimestamp;
}
public void setCompletedTimestamp(String completedTimestamp) {
this.completedTimestamp = completedTimestamp;
}
}