Topic: IBM scans ten billion files during your lunch hour

[Simon]

IBM has unveiled a single system that can scan 10 billion files in only 43 minutes.

If that doesn't sound impressive, keep in mind IBM's last record was one billion files scanned in three hours in 2007.

IBM made the achievement with its General Parallel File System (GPFS) - commonly used by top supercomputers - running on a cluster of ten eight-core systems paired with 6.8 terabytes of solid-state storage.

"GPFS's advanced algorithm makes possible the full use of all processor cores on all of these machines in all phases of the task (data read, sorting and rules evaluation)," IBM explained.

"The appliances sustainably perform hundreds of millions of data input-output operations, while GPFS continuously identifies, selects and sorts the right set of files among the 10 billion on the system," IBM said.

The breakthrough makes it possible to hold huge amounts of data on a single platform, while keeping it manageable.

Read more: http://www.pcpro.co.uk/news/368830/ibm-scans-ten-billion-files-during-your-lunch-hour

[Rik]

I want one, I want one now! 

[Simon]

I'd never heard of a 'Zettabyte'.  I wonder who thinks of the names of these, and how far they actually go up?

[Rik]

There's not many letters after Z...

[Simon]

True. 

[GillE]

There's not many letters after Z...

Oh, I don't know.  On my keyboard there's xcvbnm,./

 

[Simon]

[Rik]