VARNISH CACHE V/S CDN

Varnish Cache

Varnish is a reverse HTTP proxy, sometimes referred to as a HTTP accelerator or a web accelerator. It stores files or fragments of files in memory, allowing them to be served quickly. It is essentially a key/value store that usually uses the URL as a key. It is designed for modern hardware, modern operating systems and modern workloads.

Sitting in front of your website, Varnish Cache deals with serving static or quasi-static content directly, (It can cache certain static elements, such as images or JavaScript but you can also use it for other purposes such as Load balancing or some additional security.) without passing the request back to the web server (i.e. Apache) to process over and over again. Since a lot of content (despite the amount of time they will be accessed) needs to be computed and generated only once, storing and then serving them from the fast access memory greatly reduces the load bore by the web server and increases the amount of requests that can be simultaneously handled by our system as a whole.

To Note: Unless we use a separate application server instance for Varnish (which we should, for many good reasons!) this is technically not scaling (neither up, nor out), but having a good system architecture and basically being smart. However, due to the increase of load that can be handled by a single server, this could be referred to as scaling in some ways, as we are going to have much better overall performance from our system. This is really the result expected from scaling, and we end up with a system architecture that is ready to truly scale if necessary; however, it must be noted that better performance alone does not mean scaling.

 

At the same time, Varnish is flexible. The Varnish Configuration Language is lightning fast and allows the administrator to express their wanted policy rather than being constrained by what the Varnish developers want to cater for or could think of. Varnish has shown itself to work well both on large (and expensive) servers and tiny appliances.

Properties of Caching Reverse Proxy

A caching reverse proxy server typically has these properties:

  • Local (close) to the web application
  • Reduces the load on web/application servers for cacheable content
  • Can cache many kinds of content, including dynamically-generated content
  • Full control of cache flushes is with you
  • The web application is ‘unaware’ of the caching taking place and does not need to be  modified for the benefits of reverse proxy caching to be had

 

CDN (Content Delivery Network):

A content delivery network (CDN) is a system of distributed servers (network) that deliver webpages and other Web content to a user based on the geographic locations of the user, the origin of the webpage and a content delivery server. This service is effective in speeding the delivery of content of websites with high traffic and websites that have global reach. The closer the CDN server is to the user geographically, the faster the content will be delivered to the user. CDNs also provide protection from large surges in traffic.

Properties of a CDN

A CDN typically has the following properties:

  • A set of “edge” servers which are located in various distinct geographic locations
  • Suitable for slowly-changing content, because content propagation times are relatively high (hours)
  • Owned by a third party
  • Usually combined with custom DNS solutions (with low DNS TTL values) to effect the geo-direction
  • Disconnected (by design) from the web application
  • Typically serve “static” content such as images, Flash, video, etc.
  • Cannot effectively cache dynamically-generated content
  • URLs or applications often need to be modified to work with the CDN

Where to use Varnish Cache:

  • If you need to reduce load on web or application servers for common content, use a caching reverse proxy.
  • A caching reverse proxy reduces load on web/application servers and avoids unnecessary trips to a database or other content store for frequently-accessed content.

 

Where to use CDN:

  • If your users are geographically spread, use a CDN.
  • A CDN locates static content geographically close to end-users to avoid transmission delay

 Recommendation:

  • If you need to address above mentioned issues, use both a CDN and Varnish as a caching reverse proxy.
  • Varnish can sit in front of your site as an act as the caching gate keeper. A pull CDN will cache content/assets on the edge of the network. When a CDN node doesn’t have a copy of the page or asset it will call back to the origin (aka your server) for that item. If you don’t have Varnish in place and are using a CDN with many POPs (points of presence) your server will still get hammered with requests when adding new content.

Difference Between Queues and Topics-JMS

Queues

JMS queues is a staging area that contains messages that have been sent and are waiting to be read. Queue represent PTP domain. A point-to-point (PTP) product or application is built around the concept of message queues, senders, and receivers. Each message is addressed to a specific queue, and receiving clients extract messages from the queue(s) established to hold their messages. Queues retain all messages sent to them until the messages are consumed or until the messages expire.There can be more but the message will be received to one of them. It also guarantees delivery of the message, means message will be there in the queue unless it is consumed by someone.

TOPICS

Topic represent Publish/Subscibe domain. A distribution mechanism for publishing messages that are delivered to multiple subscriber. In a publish/subscribe (pub/sub) product or application, clients address messages to a topic. Publishers and subscribers are generally anonymous and may dynamically publish or subscribe to the content hierarchy. The system takes care of distributing the messages arriving from a topic’s multiple publishers to its multiple subscribers. Topics retain messages only as long as it takes to distribute them to current subscribers.

Publish/Subscribe Model

The publish/subscribe model supports publishing messages to a particular message topic. Subscribers may register interest in receiving messages on a particular message topic. In this model, neither the publisher nor the subscriber knows about each other. A good analogy for this is an anonymous bulletin board.

The following are characteristics of this model:

  1. Zero or more consumers will receive the message.
  2. There is a timing dependency between publishers and subscribers. The publisher has to create a message topic for clients to subscribe. The subscriber has to remain continuously active to receive messages, unless it has established a durable subscription. In that case, messages published while the subscriber is not connected will be redistributed whenever it reconnects

Some Tips To Improve Performance Of Your Web-Application


What Slows your site down? (Most expensive to least)

  1. Database write access (read is cheaper)
  2. Database read access
  3. PHP, ASP, JSP and any other server side scripting
  4. Client side JavaScript
  5. Multiple/Fat Images, scripts or css files from different domains on your page
  6. Slow keep-alive client connections, clogging your available sockets

How to make your site run faster

  • Database layer

  1. Switch all database writes to offline processing.
  2. Minimize number of database read access to the bare minimum. No more than two queries per page.
  3. De-normalize your database and Optimize MySQL tables
  4. Implement MemCached and change your database-access layer to fetch information from the in-memory database first. The least you should do is store all sessions in memory.
  5. If your system has high reads, keep MySQL tables as MyISAM. If your system has high writes, switch MySQL tables to InnoDB. If you need 99.999% availablity – consider switching to MySQL Cluster storage.
  • Server side scripting

  1. Limit server side processing to the minimum.
  2. Short scripts that queue any heavyduty processing to be done offline. Use a caching engine that generates static files from dynamic ones, so that processing only takes place once.
  3. Per-compile all php scripts using eAccelrator. If you’re using WordPress, implement WP-Cache
  • Front end

  1. Reduce size of all images by using an image optimizer, Merge multiple css/js files into one, Minify your .js scripts
  2. Avoid hardlinking to images or scripts residing on other domains.
  3. Put .css references at the top of your page, .js scripts at the bottom.
  4. Install FireFox FireBug and YSlow. YSlow analyze your web pages on the fly, giving you a performance grade and recommending the changes you need to make.
  • Web Server

  1. Optimize httpd.conf to kill connections after 5 seconds of inactivity, turn gzip compression on.
  2. Configure Apache to add Expire and ETag headers, allowing client web browsers to cache images, .css and .js files
  3. Consider dumping Apache and replacing it with Lighttpd or Nginx. If you absolutely need to stick with Apache – upgrade to its latest version.
  • The Results

  1. Implementing the steps described above will result in a faster browsing experience for your visitors as well as significantly improved website scalability.
  2. The chart below illustrates heavy load impact on a non optimized machine (linear degradation in performance) vs an optimized machine.
  3. With our non optimized machine, CPU spiked to 90% with 50 concurrent connections. The optimized machine was effectively handling 500 concurrent connections per second with CPU at 8% and no degradation in performance.

Get All the Contacts From Gmail Using Google Data Protocol

How to Use Google Data Api

<html>
 <head><title>Get Authentication Token</title></head>
 <%@ page contentType="text/html;charset=UTF-8" language="java" %>
 <body>
  	<b>Get the Authentication  url from the google : </b><a href="token.do">click Here</a>
  </body>
</html>

Here is the general process to acquire an OAuth token for a given user

  1. Your registered web application makes a signed request for a request token at Google’s endpoint, https://www.google.com/accounts/OAuthGetRequestToken. Depending on the type of HTTP request being made, the required oauth_* query parameters can either be in the body of the request, as part of the Authorization header, or in the query part of the URL. In addition to the standard OAuth parameters, you must include a scope query parameter that is appropriate for the Google Data API(s) your application will interact with.
  2. Your web app sends the user to Google’s authorization endpoint, https://www.google.com/accounts/OAuthAuthorizeToken, referencing the request token and including the oauth_callback parameter. Google may prompt the user to log into their Google Account. Once authenticated with Google, the user chooses to share their data.
  3. The request token is authorized and Google redirects the user back to the URL you specified in the oauth_callback query parameter.
  4. Your web app sends a signed request to Google’s access token endpoint, https://www.google.com/accounts/OAuthGetAccessToken, to exchange the authorized request token for an access token.
  5. If the request is verified, Google responds with a valid access token.
  6. Your web app uses the access token to send signed requests to one or more of the Google Data APIs. The API(s) your application can interact with will depend on the value that was set in the initial scope query parameter.

Code For Getting auth token

</pre>
</pre>
package com.foo.bar;

import java.io.IOException;
import javax.servlet.RequestDispatcher;
import javax.servlet.ServletException;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

import com.google.gdata.client.http.AuthSubUtil;
/**
 *
 * @author RAJENDRA
 * GmailToken class is used to create the authentication token for accessing the gmail account. In this class you have
 * to define your return back action name so after authentication you can return back on your website.
 */
public class GmailToken  extends HttpServlet{

 protected void doPost(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {

 String gmailAuthKey=getAuth();
 request.setAttribute("gmailAuthKey", gmailAuthKey);
 RequestDispatcher rd = getServletContext().getRequestDispatcher("/gmailToken.jsp");
 rd.forward(request, response);
 }

 protected void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
 doPost(request, response);
 }

 /**
 * The method is used to set the action url , so after authentication you can return back on your main website.
 * getting the token from the gmail
 * @return gmailAuthenticationToken
 */
 private String getAuth() {
 String authSubLogin=new String();
 String next = "http://localhost:8081/Gmail-Contacts/friends.do";
 String scope = "http://www.google.com/m8/feeds/";
 boolean secure = false;
 boolean session = true;
 authSubLogin = AuthSubUtil.getRequestUrl(next, scope, secure, session);
 return authSubLogin;
 }

}</pre>
<pre>
<pre>
<pre>
<pre>

Using Auth Token Access Gmail Api

After a successful authentication request, use the Auth value to create an Authorization header for each request to  the Google Data API:
Authorization: GoogleLogin auth=yourAuthToken

<%@ page contentType=”text/html;charset=UTF-8″ language=”java” %> <html> <head> <title> Validate-the-email </title> </head> <body> <b>Validate the email id on the google account</b> <a href='<%=request.getAttribute(“gmailAuthKey”)%> />’>GMAIL</a> </body> </html>

package com.foo.bar;

import java.io.IOException;
import java.net.URL;
import java.security.GeneralSecurityException;

import javax.servlet.RequestDispatcher;
import javax.servlet.ServletException;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

import com.google.gdata.client.contacts.ContactQuery;
import com.google.gdata.client.contacts.ContactsService;
import com.google.gdata.data.contacts.ContactFeed;
import com.google.gdata.util.ServiceException;
/**
 *
 * @author RAJENDRA
 * The GetAllContact class is used to validate and get all the contact list from your gmail.
 */
public class GetAllContact extends HttpServlet {

	public void doPost(HttpServletRequest req, HttpServletResponse res)throws ServletException, IOException {
		String token = req.getParameter("token");
		try {
			ContactFeed contactFeed = getContactList(token);
			req.setAttribute("contactFeed", contactFeed);
		} catch (GeneralSecurityException e) {
			e.printStackTrace();
		} catch (ServiceException e) {
			e.printStackTrace();
		}
		RequestDispatcher rd = getServletContext().getRequestDispatcher(
				"/showAllContacts.jsp");
		rd.forward(req, res);
	}

	public void doGet(HttpServletRequest req, HttpServletResponse res)throws ServletException, IOException {
		doPost(req, res);
	}

	/**
	 * In this api we are setting the authentication token for the particular
	 * user and calling the google service
	 * @param token
	 * @return
	 * @throws IOException
	 * @throws GeneralSecurityException
	 * @throws ServiceException
	 */
	public ContactFeed getContactList(String token) throws IOException,GeneralSecurityException, ServiceException {
		ContactFeed contactFeed = null;
		try {
			ContactsService contactsService = new ContactsService(
					"GoogleInc-jsguide-1.0");
			contactsService.setAuthSubToken(token);
			contactFeed = getAllContacts(contactsService);
		} catch (Exception e) {
			e.printStackTrace();
		} catch (Error error) {
			error.printStackTrace();
		}
		return contactFeed;
	}

	/**
	 * Set other information regarding the email list and get the contact
	 * feed list
	 * @param myService
	 * @throws ServiceException
	 * @throws IOException
	 */
	public ContactFeed getAllContacts(ContactsService myService)
			throws ServiceException, IOException {
		URL feedUrl = new URL(
				"http://www.google.com/m8/feeds/contacts/default/full");
		ContactQuery contactQuery = new ContactQuery(feedUrl);
		contactQuery.setMaxResults(1000);
		ContactFeed resultFeed = myService.getFeed(contactQuery,
				ContactFeed.class);
		return resultFeed;
	}

}

Show All The Contacts On Jsp

<%@page import="com.google.gdata.data.contacts.ContactFeed"%> <html><head><title>Show all the email in my account</title></head><%@page import="com.google.gdata.data.contacts.ContactEntry" %><%@page import="com.google.gdata.data.extensions.Email"%><body><table>



<h1>ALL THE CONTACT FROM YOUR GMAIL CONTACT LIST<h1>
<%ContactFeed resultFeed = (ContactFeed)request.getAttribute("contactFeed");
for (int i = 0; i < resultFeed.getEntries().size(); i++) {
 ContactEntry entry = resultFeed.getEntries().get(i);
  for (Email email : entry.getEmailAddresses()) {
       %><tr><td><b>Email add:</b></td><td><%=email.getAddress()%></td>
<%if (email.getRel() != null) { %> <% }if (email.getLabel() != null) {
 %><td><b>label</b></td><td> <%=email.getLabel()%></td></tr>
<% }if (email.getPrimary()) {}
%></table></body>
</html>


Access code from the google svn:

http://gmailcontact.googlecode.com/svn/trunk/





Installation of mysql on the linux machine

Many time I faced a problem to install mysql on the linux operating system,  So in this blog I mentioned  installation step of  mysql on  linux machine.

Get the mysql installation  from http://ftp.jaist.ac.jp/pub/mysql/ site

  1. wget http://dev.mysql.com/get/Downloads/MySQL-5.1/mysql-5.1.41-linux-i686-glibc23.tar.gz/.
  2. Unzip the mysql-5.1.41-linux-i686-glibc23.tar.gz file.
  3. Goto the mysql dir, and run the following commands
groupadd mysql
useradd -g mysql mysql
chown -R mysql .
chgrp -R mysql .
scripts/mysql_install_db  --user=mysql
chown -R root .
chown -R mysql data
bin/mysqld_safe --user=mysql &
bin/mysqladmin -u root password  'password'

Read more of this post

Microsoft DOS netstat command to displays all local network information

For Windows

Use netstat -na -b  to see names of programs along with the port they are using :
Windows command :netstat -na -b

netstat command for windows

For Linux

Use  netstat -nlp to see PID and names of programs along with the port they are using (note you can only see the PID of process you own, unless running as root)

Linux command :netstat -nl

Java Install Builder

Few month before I am searching for free multi-platform Java installer builder. But we didn’t find any free java installer builder then finally we move on to the “Install4j“. We worked on the install4j around 3-4 month. We found that for the basic functionalities(for few screens and for some action) install4j worked fine. But when we are building some complex installer that time install4j have lot of issues. The most common issues with the install4j which I faced is Dynamically bundle of JRE. When I dynamically bundle the jre I am able to bundle the jre, but once the jre bundled with the installer it always extract. There is no way to prevent the extraction of the jre. Even on the completion of the installer if I want to trigger any batch file or sh file for the deletion of the jre, the install4j didn’t allowed to delete the jre. I discussed same thing with the “Install4j” support team but I got straight forward answer that we can’t do that with the “Install4j”.

In this blog I want some support. Please provide me some other “Java install builder tool”.