Google releasing huge word usage database
Posted on August 4, 2006
On Google’s official research blog (they have a research blog? subscribed!) they report that they are going to release via LDC a huge database on the statistical occurence of word sequences.
We believe that the entire research community can benefit from access to such massive amounts of data. It will advance the state of the art, it will focus research in the promising direction of large-scale, data-driven approaches, and it will allow all research groups, no matter how large or small their computing resources, to play together. That’s why we decided to share this enormous dataset with everyone. We processed 1,011,582,453,213 words of running text and are publishing the counts for all 1,146,580,664 five-word sequences that appear at least 40 times. There are 13,653,070 unique words, after discarding words that appear less than 200 times.
This could be very useful for a lot of speech research. Link. Thanks, Google Blogoscoped.
» Filed Under google, research, speech
Comments
Leave a Reply