Chris Broekema (ASTRON) and I received funding from our national research council NWO to research a new generic AI-driven co-design method, which can generate efficient code for all sorts of different computer hardware. With more efficient code, less computing power is needed to realise data-intensive scientific research, making the process more sustainable. The official announcement can be found here. Using this grant with a total value of 933K euro, we will purchase new emerging compute hardware and can hire two new PhD students at the Leiden Institute for Advanced Computer Science (LIACS) who will research the new methods, with concrete implementations for radio astronomy codes.
This project is a public-private partnership with several companies and institutes:
- Leiden University
- ASTRON
- SURF
- CGI
- TriOpSys B.V.
- S[&]T
- The Netherlands eScience Center
- Sioux Technologies B.V.
Without the invaluable help of these partners, this research would not be possible!
Currently, data-intensive scientific applications generally require vast amounts of compute resources to deliver world-leading science. The climate emergency has made it clear that unlimited use of resources (e.g., energy) for scientific discovery is no longer acceptable. Future computing hardware promises to be much more energy efficient, but without better optimized software we cannot reach its full potential. In the SuperCode project we develop a generic AI-driven co-design methodology, using specialized large language models (like ChatGPT), to effectively generate efficient code for emerging computing hardware. We will validate our methodology with two radio astronomy applications, with sustainability as the key performance indicator.
Scientific Summary
Data-intensive science requires vast amounts of compute resources to deliver world-leading results. The current energy and environmental crises drive a strong desire to do science in a manner that minimizes the environmental impact we make while maximizing the science we can deliver. Modern special-purpose compute architectures promise to be much more power efficient than general purpose systems. However, leveraging these new architectures is time-consuming and thus expensive due to the effort it takes to port existing code to a new architecture and the increasing complexity and specialization of hardware components. We investigate how we can improve effective co-design of hardware and software since this is essential to ensure the resulting combination is fit for purpose and able to run efficiently. We hypothesize that recent advances in code generation with AI-based large language models (LLMs, e.g., ChatGPT) can be a catalyst for this process. We propose a systematic AI-driven co-design methodology that can drastically reduce the turn-around time to evaluate emerging technology for data intensive science, with sustainability as key performance indicator. To validate our novel approach, we explore two radio astronomy science cases and investigate their most optimal and sustainable emerging technology platform. With our partners, we will explore opportunities in other domains like climate research, remote sensing and earth observation.