HPC in Digital Continuum
Things have changed since we wrote our first Strategic Research Agenda in 2013. That text defined the European HPC Techology Research prorities across the various elements of the HPC System Stack without looking at any other technologies. At that time, we developed a model, which we called 'Multidimensional' and which provided the basis of HPC Technology development in four directions: System Stack, Extreme-Scale Requirements, New HPC Deployments and Use Expansion. The working groups of all our SRAs - including the latest one (SRA 4) - stem from this model.
This model lasted us till SRA 3, in which we introduced the topic of working together with Big Data and the new concept of Extreme-Scale Demonstrators.
Then, our Blueprint (2019) introduced the idea of the Digital Continuum in which HPC plays the critical role of orchestrating and enabling workflows operated by other sister technologies such as Big Data, Artificial Intelligence, Internet of Things, etc., as illustrated below.
The rapid proliferation of digital data generators, the unprecedented growth in the volume and diversity of the data they generate, and the intense evolution of the methods for analysing and using that data are radically reshaping the landscape of scientific computing. The most critical problems involve the logistics of wide-area, multistage workflows that move back and forth across the computing continuum, between the multitude of distributed sensors, instruments and other devices at the network’s Edge and the centralised resources of commercial clouds and HPC centres.
The figure above illustrates High-Performance Computing as one element of a complex workflow (“HPC in the loop"), starting with data generated at smart sensors in an IoT environment. Data is being locally pre-processed at the Edge and relevant parts are forwarded to decentralised “Fog nodes” close to the Edge. A subset of data is then transferred for centralised Data Analytics or simulation and modelling in centralised HPC centres or clouds. In an increasing number of use scenarios based on the concept of the “Digital Twin”, a “twin-copy” of a physical entity, is maintained and continuously updated on these central compute infrastructures (see chapter “Centre to Edge framework” on page 90). It should be noted that in reality the dependencies between the segments of the loop shown in the Figure are not sequential in nature. The loop is not strictly repeating actions in a circular mode; the elements are cross-connected in rather complex, often fast changing event-driven flows.
The final outcome of the loop is a set of optimised actions in the “Cyber Physical Entanglement” representing physical systems (e.g. robots, vehicles, industrial processes) interconnected in complex intelligent networks.
SRA 4 operates in the context of this Digital Continuum. Apart from the well-developed eight technical focal areas of HPC related technology, this documents also presents an argument in favour of placing HPC within the context of “The Digital Continuum spectrum”. The role HPC can play in this new concept is illustrated by four advanced use cases, which emphasise the need to master a multitude of new challenges presented by the complex workflows exemplifying this Digital Continuum.
We also introduce a new approach to defining the contents of the future research programmes. We believe that the most important challenge of European HPC now is to serve the development of the Digital Continuum, i.e. the unison of HPC and related technologies. We use the concept of Research Clusters to represent the main challenges of this Continuum. We argue that the future research should focus on how HPC can serve those areas. The priorities of the future Work Programmes could be extracted from this SRA by looking at the traditional Research Domains or Research Clusters, or by looking at their intersections, where – we believe – the core challenges lie.
This work will continue into the next SRA. We continue the work started with the other stakeholders of the European Digital arena through the TransContinuum Initiative coordinated by our association.
