Navigation

Hungry to learn: Ultrafast algorithms devour Big Data

 

The human race is producing ever larger amounts of data. Computers have the ability to learn from data, but dealing with huge volumes of it still poses a challenge. This project develops powerful, ultrafast algorithms for learning from Big Data.

Portrait / project description (ongoing research project)

“Gaussian processes”refer to a family of powerful algorithms used in machine learning. These algorithms have many virtues. They can learn from any data, no matter how complex. They have good mathematical properties for reliable predictions. They are clear and readily understandable. But this power comes at the cost of processing time that is incompatible with Big Data. Nonetheless, in some cases it is possible to approximate these algorithms to compute data of any size. This project aims at extending this capability to all cases by developing algorithms for Big Data that have the potential to transform applications.

Background

Machine learning is a field of research that creates increasingly smarter computer procedures (algorithms) for analysing Big Data. This combination of Big Data and intelligent algorithms offers an unprecedented opportunity to learn more about the world and, thus, to accelerate progress. But the sheer amount of data available also presents challenges: large amounts of complex data must be analysed in ways that are reliable and relevant, and the data must be processed efficiently.

Aim

The usefulness of an algorithm in analyzing data is limited by the speed required to process it. Much research in machine learning is thus geared towards combining power and speed, but this has yet to be achieved with Big Data. This project focuses on a new approach to creating algorithms that deliver both maximum power and speed. They enable the promise of Big Data to be fulfilled.

Relevance/application

The project involves two real-world applications for our new algorithms. The first is a collaboration with MeteoSwiss to improve the prediction of rainfall intensity from huge amounts of radar data for meteo alerting. The second collaboration, with Armasuisse, will predict electrosmog from massive amounts of sensor data. Many more problems can be addressed using the same algorithms, for example in the areas of medicine, business and science.

Original title

State space Gaussian processes for big data analytics

Project leaders

  • Professor Marco Zaffalon, Istituto Dalle Molle di Studi sull'Intelligenza Artificiale, Manno
  • Dr. Alessio Benavoli, Istituto Dalle Molle di Studi sull'Intelligenza Artificiale, Manno

 

 

Further information on this content

 Contact

Professor Marco Zaffalon Istituto Dalle Molle di Studi sull'Intelligenza Artificiale Galleria 2 6928 Manno zaffalon@idsia.ch

On this Subject