How Google Works

How Google Works

Google runs on a distributed network of thousands of low-cost computers and can therefore carry out fast parallel processing. Parallel processing is a method of computation in which many calculations can be performed simultaneously, significantly speeding up data processing. Google has three distinct parts:


• Googlebot, a web crawler that finds and fetches web pages.
• The indexer that sorts every word on every page and stores the resulting index of words in a huge database.
• The query processor, which compares your search query to the index and recommends the documents that it considers most relevant.

Let’s take a closer look at each part.
1. Googlebot, Google’s Web Crawler

Googlebot is Google’s web crawling robot, which finds and retrieves pages on the web and hands them off to the Google indexer. It’s easy to imagine Googlebot as a little spider scurrying across the strands of cyberspace, but in reality Googlebot doesn’t traverse the web at all. It functions much like your web browser, by sending a request to a web server for a web page, downloading the entire page, and then handing it off to Google’s indexer.

Googlebot consists of many computers requesting and fetching pages much more quickly than you can with your web browser. In fact, Googlebot can request thousands of different pages simultaneously. To avoid overwhelming web servers, or crowding out requests from human users, Googlebot deliberately makes requests of each individual web server more slowly than it’s capable of doing.

Googlebot finds pages in two ways: through an add URL form, www.google.com/addurl.html, and through finding links by crawling the web.



2. Google’s Indexer

Googlebot gives the indexer the full text of the pages it finds. These pages are stored in Google’s index database. This index is sorted alphabetically by search term, with each index entry storing a list of documents in which the term appears and the location within the text where it occurs. This data structure allows rapid access to documents that contain user query terms.

To improve search performance, Google ignores (doesn’t index) common words called stop words (such as the, is, on, or, of, how, why, as well as certain single digits and single letters). Stop words are so common that they do little to narrow a search, and therefore they can safely be discarded. The indexer also ignores some punctuation and multiple spaces, as well as converting all letters to lowercase, to improve Google’s performance.


3. Google’s Query Processor


The query processor has several parts, including the user interface (search box), the “engine” that evaluates queries and matches them to relevant documents, and the results formatter.

Page Rank is Google’s system for ranking web pages. A page with a higher Page Rank is deemed more important and is more likely to be listed above a page with a lower Page Rank.


I hope you like this article.


Regards,

Prateek

About AJAX

About AJAX

• AJAX stands for Asynchronous JavaScript and XML.
• AJAX is become popular with Google application like Google suggestion

AJAX Components

• HTML
• CSS
• DOM
• JavaScript
• XML
• JSon
• XMLHttp

AJAX Framework

• Built on top of ASP.NET 2.0, there is a valuable set of controls and services that extend the existing framework with Ajax support.
• This tier of the server framework is called the ASP.NET AJAX Server Extensions.
• The server extensions are broken up into three areas: server controls, Web Services Bridge and the Application Services Bridge.
• Each of these components interacts closely with the application model on the client to improve the interactivity of existing ASP.NET pages.

ASP.NET AJAX Server Controls

The new set of server controls add to the already impressive arsenal of tools in the ASP.NET toolbox and are predominantly driven by two main controls.

ScriptManager - is considered to be the brains of an Ajax-enabled page. One of the many responsibilities of the ScriptManager isorchestrating the regions on the page that are dynamically updated during asynchronous postbacks.

UpdatePanel - is used to define the regions on the page that are designated for partial updates.

Getting the IP address

Well, yes, it looks like trivial but it isn't. Many a times we need to get the IP address of the user. There are a handful of suggestions to do this. I discuss the methods and the short comings in them.

1. REMOTE_ADDR


string ipAddress = Request.ServerVariables["REMOTE_ADDR"];

This method fails when the user's request travels through a Proxy server. The Ip address in this case will be of the proxy.

2. HTTP_X_FORWARDED_FOR


string ipAddress = Request.ServerVariables["HTTP_X_FORWARDED_FOR"];

Generally method 1 is overcome by using HTTP_X_FORWARDED_FOR server variable. This method returns the user address. Well this method fails when the user request does not come through a proxy.

3. REMOTE_ADDR and HTTP_X_FORWARDED_FOR


string ipAddress = Request.ServerVariables["HTTP_X_FORWARDED_FOR"];
if (string.IsNullOrEmpty(ipAddress))
ipAddress = Request.ServerVariables["REMOTE_ADDR"];

Using the combination of method 1 and 2 we come to method 3. Do you think this is the solution? No. This ain't too. The reason being, sometimes the requests passes through multiple Proxy servers. In this case HTTP_X_FORWARDED_FOR contains comma separated values of the various proxy servers through which the request passes. So you actually do not have one IP address but multiple ones.

4. REMOTE_ADDR and HTTP_X_FORWARDED_FOR and some processing


string ipAddress = Request.ServerVariables["HTTP_X_FORWARDED_FOR"];
if (string.IsNullOrEmpty(ipAddress))
ipAddress = Request.ServerVariables["REMOTE_ADDR"];

int index = -1;
if ((index = ipAddress.IndexOf(",")) > -1)
{
ipAddress = ipAddress.Substring(0, index);
}

The final solution we come to is, use method 3 but check if there is comma separated values. If there is then we take the first IP address which will be the IP address of the user.

I have formatted this into a method which can be reused,



public static class Utility
{
public static string GetIP(NameValueCollection nameValueCollection)
{
string result = string.Empty;

if (nameValueCollection == null || nameValueCollection.Count < 1)
return result;

result = nameValueCollection["HTTP_X_FORWARDED_FOR"];
if (string.IsNullOrEmpty(result))
return nameValueCollection["REMOTE_ADDR"];

int index = -1;
if ((index = result.IndexOf(",")) > -1)
{
result = result.Substring(0, index);
}

return result;
}
}

Request.ServerVariables is a NameValueCollection. To use the above method you will have to pass in Request.ServerVariables as a parameter. Like this,


string ipAddress = Utility.GetIP(Request.ServerVariables);


An important inclusion is that this method works for Transparent proxies. There is no way (what I know) to detect the real IP address of a user for a Anonymous/Distorting Proxy.

Google Dance

Google Dance
The name Google Dance is a term used to describe the index update of the Google search engine. It's the period when Google is rebuilding its rankings, and results fluctuate widely for a 3 to 5 day period. It can be identified by significant movement in search results and especially by Google's cache of all indexed pages reflecting the status of Google's last spidering.

Approximately once a month, Google update their index by recalculating the Page ranks of each of the web pages that they have crawled. The period during the update is known as the Google dance.

Because of the nature of Page Rank, the calculations need to be performed about 40 times and, because the index is so large, the calculations take several days to complete. During this period, the search results fluctuate; sometimes minute-by minute. It is because of these fluctuations that the term, Google Dance, was coined. The dance usually takes place sometime during the last third of each month.

Google has two other servers that can be used for searching. The search results on them also change during the monthly update and they are part of the Google dance.

Checking the Google Dance

Until January 2004, Google had 12 main www servers online, which were as follows:

www-ex.google.com - (where you get when you type www.google.com)
www-sj.google.com - (which can also be accessed at www2.google.com)
www-va.google.com - (which can also be accessed at www3.google.com)
www-dc.google.com
www-ab.google.com
www-in.google.com
www-zu.google.com
www-cw.google.com
www-fi.google.com - found in May 2003.
www-gv.google.com - found in August 2003.
www-gv2.google.com - found in September 2003.
www-kr.google.com - found in October 2003.
At some point in January, these servers stopped accepting connections, and the only servers easy to connect to be:

www.google.com
www2.google.com
www3.google.com

Export data in excel from gridview

Here is the comple code to generate data in excel from gridview....


protected void Button1_Click(object sender, EventArgs e)
{
Response.Clear();
Response.Buffer = true;
Response.ContentType = "application/ms-excel";
Response.AddHeader("content-disposition", string.Format("file.xls", "excelsheet"));

System.IO.StringWriter stringwriter = new StringWriter();
HtmlTextWriter hw = new HtmlTextWriter(stringwriter);
GridView1.RenderControl(hw);
Response.Write(stringwriter.ToString());
Response.End();

}

Creating Add to Favorites link through JavaScript

JavaScript function code:
function addfav()
{
if (document.all)
{
window.external.AddFavorite
("http://www.bprateek.blogspot.com","Prateek Tips")
}
}

from the link as:
Add To Favorites

State Management using Querystring

//send data as querystring Response.Redirect("http://localhost:3521/Basic/StateManagement/Stateinfo.aspx?name=sysmphony-services&lang=en-US");
//Querystring information
Label1.Text = "Query string output:Name: " + Server.HtmlEncode(Request.QueryString["name"]) +
", Lang: " + Server.HtmlEncode(Request.QueryString["lang"]);