Sunday, November 30, 2008

gimme


oaknyc

drive


tfs

Wednesday, November 26, 2008

so drama


foto_decadent

why can't this be me? so cute!


tfs

Triple silken pumpkin pie takes the prize

What is the first thing that most people associate with Thanksgiving? Well, probably turkey, but pumpkin pie comes in a close second. As I am not a fan of the traditional pumpkin pie, I set out on a quest to find a delicious alternative this year. My search led me to a robust cookbook sitting on my kitchen shelves. Sheri Yard's Desserts by the Yard is an amazing compilation of a pastry chef's career spanning from coast to coast. What I found in that book turned out to be the most fluffy, decadent, flaky, scrumptious pie I have ever tasted. And apparently my officemates liked it just as much -- the triple silken pumpkin pie and I took home first place in last week's bake-off at our New York office! So if you're looking for a holiday-perfect pie, I encourage you to try out the recipe (PDF file). It takes a little time to make, but it's so worth it.

Happy baking, and happy Thanksgiving!



Friday, November 21, 2008

hot....pink


delightful

blue


CopenhagenStreetStyle

Sorting 1PB with MapReduce

At Google we are fanatical about organizing the world's information. As a result, we spend a lot of time finding better ways to sort information using MapReduce, a key component of our software infrastructure that allows us to run multiple processes simultaneously. MapReduce is a perfect solution for many of the computations we run daily, due in large part to its simplicity, applicability to a wide range of real-world computing tasks, and natural translation to highly scalable distributed implementations that harness the power of thousands of computers.

In our sorting experiments we have followed the rules of a standard terabyte (TB) sort benchmark. Standardized experiments help us understand and compare the benefits of various technologies and also add a competitive spirit. You can think of it as an Olympic event for computations. By pushing the boundaries of these types of programs, we learn about the limitations of current technologies as well as the lessons useful in designing next generation computing platforms. This, in turn, should help everyone have faster access to higher-quality information.

We are excited to announce we were able to sort 1TB (stored on the Google File System as 10 billion 100-byte records in uncompressed text files) on 1,000 computers in 68 seconds. By comparison, the previous 1TB sorting record is 209 seconds on 910 computers.

Sometimes you need to sort more than a terabyte, so we were curious to find out what happens when you sort more and gave one petabyte (PB) a try. One petabyte is a thousand terabytes, or, to put this amount in perspective, it is 12 times the amount of archived web data in the U.S. Library of Congress as of May 2008. In comparison, consider that the aggregate size of data processed by all instances of MapReduce at Google was on average 20PB per day in January 2008.

It took six hours and two minutes to sort 1PB (10 trillion 100-byte records) on 4,000 computers. We're not aware of any other sorting experiment at this scale and are obviously very excited to be able to process so much data so quickly.

An interesting question came up while running experiments at such a scale: Where do you put 1PB of sorted data? We were writing it to 48,000 hard drives (we did not use the full capacity of these disks, though), and every time we ran our sort, at least one of our disks managed to break (this is not surprising at all given the duration of the test, the number of disks involved, and the expected lifetime of hard disks). To make sure we kept our sorted petabyte safe, we asked the Google File System to write three copies of each file to three different disks.

Significantly improved handling of the so-called "stragglers" (parts of computation that run slower than expected) was a key software technique that helped sort 1PB. And of course, there are many other factors that contributed to the result. We'll be discussing all of this and more in an upcoming publication. And you can also check out the video from our recent Technology RoundTable Series.

Our international approach to search

In previous posts in this series, you have read about the challenges of building a world-class search engine. Our goal is to make Google’s search be relevant to all people, regardless of their language or country. As my colleague Amit Singhal described, we use statistical data as the basis for making sweeping algorithmic changes. Many of these changes can be rolled out across all languages we support, but in some cases the unique characteristics of each language require some algorithmic considerations and tuning. And to make things really interesting, there are cases where the same language is different across countries. Obvious examples are "color" in the U.S. vs. "colour" in the U.K., or "camião" in Portugal vs. "caminhão" in Brazil.

My name is Daphne Dembo, and my focus is improving Google's international search. This is a tough challenge, since Google search is used in many countries and languages where our engineers have little personal knowledge. Initially, the international search improvements were done by Search Quality engineers who were passionate about their languages and countries: Lina from Sweden improved our parsing of compound words in German and Swedish; Dimitra from Greece introduced diacritical support; Ishai from Israel worked on transliteration corrections for Hebrew and Arabic; Trystan from Australia created methods for identifying local search results and ranking them together with foreign ones from the same language; Alex, a bilingual Ukrainian and Russian, introduced morphological understanding of these languages. As the importance of our international search grew, we solicited help from Googlers in all our offices. Finally, we are leveraging an international network of search specialists who help us understand search within the unique combination of their language and country.

Our first step in providing search support for a language is to train our language model on a large collection of documents in that language. This ensures that our language model is more precise and comprehensive — for example, it incorporates names, idioms, colloquial usage, and newly coined words not often found in static dictionaries. For instance, we recently started identifying Swahili, and used pages such as this one for the Parliament of Tanzania to train our system with the language's nuances. Having a trained language model helps to categorize documents during crawling and indexing of the web and to parse the user's query. Once this stage was complete, we launched Swahili search in countries such as Tanzania and Kenya, enabling local searches for the "Dar es Salaam stock exchange" [Soko la hisa dar es salaam], and "cure for Malaria" [Tiba ya malaria]. (As always, we are using square brackets to denote a search query. For example, you can search for "soccer" in Hamburg, Germany by clicking on [fußball in hamburg]).

We learn some things from our users, so as people start using our search engine, we can improve the way we rank in that language. Here are few examples:
  • Spell corrections: We recently launched spell corrections in Estonian. If your Estonian is rusty, and you don't remember how to spell "smoke detector," we can suggest a spell correction for [suitsuantur], leading to better search results.
  • Diacritical marks: Many languages have diacritical marks, which alter pronunciation. Our algorithms are built to support them, and even help users who mis-type or completely ignore them. For example, if you're a resident of Quebec, Canada and would like to know the weather forecast in Quebec City, we'll serve good results whether you type with diacritical signs [Météo à Québec] or without [meteo quebec]. Czech users can read the same excellent results for a popular kids' cartoon by searching for [krtecek] and [krteček]. On the other hand, sometimes diacriticals change the meaning of the word and we have to use them correctly. For example, in Thai, [ข้าว] is "rice," with completely different results than [ข่าว], which is "news"; or in Slovakia, results for "child" [dieťa] are different than results for "diet" [diéta].
  • Synonyms: A general case of diacritical support is the handling of synonyms in different languages. Korean searches showed that "samsung" can be viewed as a synonym of "삼성", so that when users search for [samsung], they find results which have the company's name in Korean.
  • Compounding: Some languages allow compounding, which is the formation of new words by combining together existing words. You can see a nice example in Swedish, where we return documents about a Swedish credit card for both compounded [Visakort] and non-compounded [visa kort] queries.
  • Stemming: Google has developed morphological models that can receive compound words as queries, and return pages which contain their stem, possibly as part of a different compound. For example, when searching for cars in Saudi Arabia, you can search for [سيارة] and [سيارات] because both are variants of the same stem, and both return many common results. A Polish user can search for "movie" [film], and get back results that contain other variants of the stem, such as "filmów," "filmu," "filmie," "filmy." A user from Belarus will find results for all word forms of the capital, Minsk [Мінск]: "Мінску," "Мінска," "Мінскага."
In addition to these semantic factors, Google does even more to parse documents and queries. Understanding the details of language usage in a country is important. Notation of acronyms is different across languages: In Hebrew it is double quotes before the last (left-most) character, as in "prime minister" [רה"מ]; in Thai — a dot at the end of the word, as in police station [สน. ]; while in the U.S. — dots after each character, as in [I.B.M.]. Chinese users quote works of art with a "《", as in: [《手机》剧情], and denote dates with a "日", as in: [2006年1月13日].

Beyond the linguistic elements of a language, we consider how people enter a query. For example, some languages that do not have Latin scripts require keyboards with dual alphanumeric keys. The user can switch between language input modes by typing special keystrokes. In case the user forgets to type this sequence, the queries end up being gibberish. You can see correct handling of these mistakes in Arabic ([hgsuv] corrected to [السعر]) and ([حقثسهيثىفهشم ثممثؤفهخىس ] corrected to [presidential elections]), Hebrew ([vdrk, kuyu] corrected to [הגרלת לוטו]), and Cyrillic ([rehc ljkffhf] corrected to [курс доллара]).

Another way of avoiding the inconvenience of switching keyboard modes is by typing the phonetic sounds of the query in Latin characters. Recreating the correct query in the target language isn't trivial, since there might be many possibilities. We can see several such examples in which we suggest the same query in the intended language for Russian ([biskvitnyi rulet] to [бисквитный рулет]), "movies" in Chinese ([dianying] to [电影]), and "Bank of Attica" in Greek [trapeza attikhs] returns good results for "Τράπεζα Αττικής". Users of 8 Indic languages (such as Hindi, Gujarati, Telugu) can type the phonetic sound of the query, and choose the words in Hindi script:


Ease of typing and reading is also influenced by the language used. Since every Chinese word requires several keystrokes on a standard keyboard, we provide category browsing by Images and related searches so that people don't need to type as much. Similarly, we are now launching Google Suggest, or real-time completion of queries, in many languages.

So far I described how we improve the quality of search in a language. However, there is a strong effect of the location of the user, even if it is only approximated to the country, since in many cases local content is more relevant than global information. For example, searching for Spanish Yellow Pages [Páginas Amarillas] will result in several documents of global interest and several local results in Peru, Mexico, and Spain. Similar to that, searching for [Côte d'Or] in France will return results for that region, whereas searches in Belgium will return results about the chocolate maker.

Note that the display of information should conform to the standards in that country, so we display "," as a decimal notation for Croatian users who want to know how many millimeters are in an inch [inč u milimetrima], or for Italian users who are interested in currency exchange rates [50 euro in dollari]. Similarly, temperatures in Norway [Været i Oslo] will be displayed in Celsius, while in the U.S. — in Fahrenheit [weather Boston].

If everything else fails, we provide cross-language translations based upon Google's translation technology described in this blog post. We will translate your query to English, search English documents on the web, and translate the returned results from English back into the original query language. For example, Japanese users who are interested in viewing Halloween illustrations (Halloween is a holiday which originated in Ireland) can search for [ハロウィン イラスト]. You can then request a Japanese translation of the English pages (at the bottom of the page), which will bring up the translation page in the screenshot below. Similarly, Korean users can search for the latest on Harry Potter [해리 포터], and Arabic readers can search for the opening of the Sydney Opera house [افتتاح دار الاوبرا في سيدني]. (Click on the image to see a larger version.)



All in all, Google Search is being actively developed for more than 100 languages, in 150+ countries, with dozens of improvements launched each month. So far I've covered the basics of how international search works, but this is just the surface of all the international work we do. There are many other interesting topics that impact international markets like usability, homepage and results page layout, and connectivity. An understanding of real cultural and human factors is essential to creating a search engine that resonates with the people who use it. (Click on the image to see a larger version.)



(Update: Replaced example in the 4th bullet point.)

it's friday! have a fabulous + stylish weekend!


ru_glamour

Blog Crush!


^click to visit

simply wonderful and very inspiring :) look at that closet!

Thursday, November 20, 2008

city streets make my heart beat


calikartel

...


thefashionspot

SearchWiki: make search your own

Have you ever wanted to mark up Google search results? Maybe you're an avid hiker and the trail map site you always go to is in the 4th or 5th position and you want to move it to the top. Or perhaps it's not there at all and you'd like to add it. Or maybe you'd like to add some notes about what you found on that site and why you thought it was useful. Starting today you can do all this and tailor Google search results to best meet your needs.

Today we're launching SearchWiki, a way for you to customize search by re-ranking, deleting, adding, and commenting on search results. With just a single click you can move the results you like to the top or add a new site. You can also write notes attached to a particular site and remove results that you don't feel belong. These modifications will be shown to you every time you do the same search in the future. SearchWiki is available to signed-in Google users. We store your changes in your Google Account. If you are wondering if you are signed in, you can always check by noting if your username appears in the upper right-hand side of the page.

The changes you make only affect your own searches. But SearchWiki also is a great way to share your insights with other searchers. You can see how the community has collectively edited the search results by clicking on the "See all notes for this SearchWiki" link.

Watch our lead engineer, Amay, demonstrate a few ways to use SearchWiki in this short video:



This new feature is an example of how search is becoming increasingly dynamic, giving people tools that make search even more useful to them in their daily lives. We have been testing bits and pieces of SearchWiki for some time through live experiments, and we incorporated much of our learnings into this release. We are constantly striving to improve our users' search experience, and this is yet another step along the way.

Transgender Remembrance Day

We believe great ideas can come from anywhere and everyone. And we aspire to be an organization that reflects global diversity, because we know that a world's worth of perspectives, ideas and cultures leads to the creation of better products and services. We have more than a dozen employee-driven resource groups, from Gayglers to GWE (Google Women Engineers), that actively participate around the world in building community and driving policy at Google. The post below kicks off an occasional series, entitled Interface, about valuing people's similarities and differences in the workplace. For more information on how Google fosters an inclusive work environment, visit Life at Google on our Jobs site. – Ed.

November 20th marks Transgender Remembrance Day, which takes on a special significance in a world awakening to the need for unity among all people. In observing this day, the Gayglers — the Lesbian, Gay, Bisexual and Transgender (LGBT) group within Google — extend their wholehearted support to the LGBT community at large, as we reflect on the senseless violence perpetrated against transgender people around the world.

People who identify or express their gender differently than the one assigned to them at birth usually call themselves transgender or transsexual. All too often, they are subjected to a range of not-so-subtle prejudices and transphobia, from verbal abuse to physical violence. Imagine walking into a public restroom in a state of dread over a confrontation about your appearance. Imagine visiting a doctor and worrying about how far to "out" yourself to receive appropriate care. Studies suggest that transgender people are 16 times more likely to be killed than the general population -- earlier this month, in fact, a transgender woman in Tennessee was murdered -- and this is just the most recent of many such cases.

We're fortunate here at Google, where there are LGB and T people at all levels of the company, thanks to enlightened hiring and promotion practices that set aside sexual orientation or gender presentation. Ultimately, Google fosters a workplace where everyone has the ability to be themselves at work. For transgender employees in particular, that means everything.

On this Transgender Remembrance Day, take just a few moments to remember the trans siblings, parents, friends and lovers who lost their lives to gender-based intolerance and hatred. Let's all share in a future where tolerance and understanding transforms the world. And let's work to create a better place for everyone to live peaceably in an all-inclusive world community that merits our deepest pride.

Wednesday, November 19, 2008

Lively no more

In July we launched Lively in Google Labs because we wanted users to be able to interact with their friends and express themselves online in new ways. Google has always been supportive of this kind of experimentation because we believe it's the best way to create groundbreaking products that make a difference to people's lives. But we've also always accepted that when you take these kinds of risks not every bet is going to pay off.

That's why, despite all the virtual high fives and creative rooms everyone has enjoyed in the last four and a half months, we've decided to shut Lively down at the end of the year. It has been a tough decision, but we want to ensure that we prioritize our resources and focus more on our core search, ads and apps business. Lively.com will be discontinued at the end of December, and everyone who has worked on the project will then move on to other teams.

We'd encourage all Lively users to capture your hard work by taking videos and screenshots of your rooms.

where is the weekend?!?!


my motto for the day

The Final Inch

(Cross-posted from the Google.org blog)

Early readers of the Google.org blog may recall us embarking on a film project portraying public health heroes working in the field to eradicate polio. Gone from the modern world, new cases of polio continue to afflict mostly children under age 3 in the poorest regions of just a few countries — India, Nigeria, Afghanistan and Pakistan. When we first announced this project and the collaboration between Google.org and Vermilion Films, filming was underway primarily in India and Afghanistan, documenting the front lines of public health in some forgotten corners of our world.

David Heymann of the World Health Organization reminds us, "When you haven't seen a disease for quite a while, which is the case in the industrialized countries, you forget about the terrible disease that it really is." Polio is such a disease, as it can ruin the lives of children even before they are old enough to understand how to prevent it.

We're proud to announce The Final Inch, a 38-minute film about the historic global effort to eradicate polio. Here, the story told is as much about the messengers as the message. You'll meet Munzareen Fatima, one of the thousands of community "foot soldiers" across India working to sway reluctant families to vaccinate their children, and Dr. Ashfaq Bhat, who travels into the backwaters of India's Ganges Basin by boat and foot to detect emerging cases of polio. Martha Mason and Mikail Davenport bring us into their lives and describe the paralyzing challenges of childhood polio, reminding us how endemic polio once was in the United States.

Filmed in high-definition (HD) in cinematic style — wide open shots to give a strong sense of place — The Final Inch captures their stories, and we hope it is both a tribute and an inspiration of hope. With a final push, this is a disease that can, and should, be eradicated finally.

The Final Inch will air nationally on HBO in 2009. We invite you to check out TheFinalInch.org, where you can view clips from the film and learn more about the people and the organizations tirelessly working on this global effort. You can also check out the film trailer here:

Update on 1/22: The Final Inch received an Oscar nomination in the category of Best Documentary (Short Film). Check out our post on the Google.org blog for more information.

Update on 3/31: The Final Inch is coming to a TV screen near you. You can tune into HBO2 on Wednesday, April 1 at 8:00pm ET/PT to see our film. It will re-air on April 7 at 7:15pm ET/PT and is available on HBO on demand. You can check out the full schedule on HBO's website.