Amalur: Next-generation Data Integration in Data Lakes

Data science workflows often require extracting, preparing and integrating data from multiple data sources. This is a cumbersome and slow process: most of the times, data scientists prepare data in a data processing system or a data lake, and export it as a table, in order for it to be consumed by a Machine Learning (ML) algorithm. Recent advances in the area of factorized ML, allow us to push down certain linear algebra (LA) operators, executing them closer to the data sources. With this work, we revisit classic data integration (DI) systems and see how these fit into modern data lakes that are meant to support LA as a first-class citizen.

• Paper [pdf]
• Talk [video]

• Bibtex:

  @inproceedings{hai2022amalur,
  title={Amalur: Next-generation Data Integration in Data Lakes},
  author={Hai, Rihan and Koutras, Christos and Ionescu, Andra and Katsifodimos, Asterios},
  booktitle={12th Annual Conference on Innovative Data Systems Research, CIDR 2022},
  year={2022}
  }