WP5 DTO Modeling and Analytics services

This task concerns the establishment of a service layer capable of receiving processing requests, submitting them to existing computing facilities along with the necessary parameters and making the results available to users in an efficient manner. Activities will include the analysis of the data processing requirements and the assessment of the expectations for each service deployment. The use of Infrastructure-as-a-Service (IaaS) to enable data and resource sharing (and bursting) ensures optimised costs and allows for massively scalable ICT infrastructure. Service agreements will be established with the data-processing infrastructure where the applications are deployed (Copernicus Data and Information Access Services (DIAS) resource providers or transparently burst on private or other commercial Cloud infrastructures) according to the needs identified in collaboration with the services. Support research grants like the Open Clouds for Research Environments (OCRE) from the European Open Science Cloud (EOSC) and the Network of Resources (NoR) initiative from the European Space Agency (ESA) will be evaluated during the project.

These initiatives aim to stimulate the usage uptake of commercial digital services by the European research community. They will provide the European research community access to commercial digital services (IaaS, SaaS and PaaS cloud services), as well as Earth Observation (EO) services. The result of this task is an architectural layer capable to enable users to select inputs from the interoperable digital twin layer (WP4), specify parameters, eventually combine them with additional data existing in other federated data spaces (e.g., observational data) and submit them to computing facilities (e.g. cloud platforms and/or HPC) for processing. When the processing is complete, results will be delivered to the data space selected by the user along with processing reports. Furthermore, the AI/Analytics processing services layer shall be duly integrated with an Authentication and Authorisation Infrastructure (AAI) supporting commonly used federated identity providers as defined in task T4.1.

• Duration:  M13 - M36
• Task Lead:  INESC TEC - Instituto de Engenharia de Sistemas e Computadores, Tecnologia e Ciência (INESC)

Usually, modelling can provide significant progress which concerns a better understanding of a local dynamic (hydrodynamic, waves patterns, water quality, etc.) through a combination of increasing model resolution, advances in observations and processes understanding. To this end there will be considered several open-source widespread meteorological, hydrodynamic, sea state, and biogeochemical models and adapt them to the Iliad DTO platform and DT specific needs. These models will be made available both in on-demand mode for a test/survey with specific temporal and spatial resolution and as to support the implementation of operational monitoring and prediction services. The geographical areas and the number of service runs will make use of existing cloud and/or HPC resources interconnected in the DTO system. Preference will be given to open-source tools, with which Iliad partners have accumulated experience, e.g. POM, Telemac3D, MOHID, Thetis, NEMO, for hydrodynamics, SWAN, Wavewatch III, WAM, OceanWave3D for wave modelling, and WRF for meteorological forecasting, or a coupling of them. Others may be investigated and integrated during the Iliad project, according to further specific requirements of the DTs. Calibration, validation, and performance assessment of these toolsfor the DTs, when integrated in Iliad platform will be thoroughly performed.

• Duration:  M7 - M30
• Task Lead:  Hidromod - Modelação em Engenharia, Lda. (HIDROMOD)

The goal of this task is to enhance the quality of numerical models by a) assimilating in-situ data; and b) incorporating synthetic data generated by surrogate models. It involves data assimilation and harmonisation of EO and in-situ data collected within the Iliad project to minimise forecasting errors and optimise operational alerting. Data assimilation techniques through either the automated calibration of models or in the continual utilisation of in-situ data will be used to perform state updating of models in both hind-casting and real-time operational models. By combining these various sources of data and properly accounting for the confidence intervals within each, the overall performance and predictive capabilities of models can be increased. There are multiple data assimilation algorithms to be considered within Iliad which include but are not limited to Ensemble Kalman filter, 3D- and 4D- variational assimilation, andParticleFiltering. Each has its own preferential implementation, depending on the temporal and spatial scale, as well as the schematisation of the model. Furthermore, Iliad will implement a surrogate approach offering both added-value information, fast and high-fidelity results for decision-making and marine spatial planning. Based on these, the integrated approach of “Hybrid digital Twin” will be developed in which numerical models, data provided by sensors (including satellite data) and synthetic data (generated by Artificial Neural Network (ANN), Gaussian Processes and other statistical models) will be combined. In case of lack of data, an integrated ANN–Kalman hybrid model and similar methods will be implemented for reconstruction of missing variables. The data synthesis approach proposed is particularly useful for nonlinear problems with large sample size where the underlying governing laws are unknown or hard to describe. This task will contribute to improving sub-grid scale processes of data and model approaches and this knowledge will be crucial to larger scale models of the Iliad DTO.

• Duration:  M14 - M30
• Task Lead:  Wave/Offshore Renewables - Centro de Energia Offshore Associacao ( WavEC)

This Task will use thematic models of specific variables (of interest to smaller groups of users), which are needed for simulations on-demand, triggered by specific events, based on a combination of user input and data from the other Iliad models. It will couple these thematic models to the operational modelling chain of DTs in order to make full use of the high-resolution data that are produced for thePCT. Thematic modelling in Iliad will include: (a) Oil spill modelling to aid the emergency response and the development of contingency plansfor accidentalsurface and subsurface oil releases, which will the configuration of models such as MEDSLIK-II, OpenOil, OSCAR; (b) Wind, current, and wave energy modelling to assess the potential for renewable energy in selected areas using Telemac3D and or Thetis; (c) Micro and macro plastics transport and distribution modelling, employing models such as OpenDrift; (d) Coastal morphodynamics modelling, (e) Ecosystem status assessment (e.g., using Ecopath with Ecosim) utilising abiotic data from the Iliad DTO, and data limited single species assessments applying the novel CMSY++ and AMSY methods; (f) fish and mussel farm modelling, by employing individual-based and population dynamics models; (g) locality-specific modelling (based on in-situ and EO data, such as wind, waves, visibility and temperature) for safe navigation in ports and channels during unfavourable weather conditions; (h) “same area” ballast water risk assessment and management. Activities of this task also include the validation of thematic models with observations.

• Duration:  M9 - M30
• Task Lead:  idryma technologias kai erevnas (FORTH)

This task researches and delivers different ML methods for pattern discovery in historical datasets and (near) real time data streams in order to infer tendencies, seasonal patterns and find deviations and anomalies, correlating and comparing data sources to understand the root cause of outlier events, developing predictive models for the early detection in repeated events of interest. Complex pattern mining is a well-known methodology for detecting behaviours indicative of recurring events of interest in spatio-temporal data streams originating from a large set of distributed sensors and remote systems (drones, satellites). However, pattern discovery can be a very time-consuming task for human experts. In this task we will make use of appropriate ML and neurofuzzy techniques, including deep neural networks (DNN), to detect events of interests and to infer indicative patterns in an unsupervised fashion. Unsupervised clustering will be used to discover clusters of patterns, which could help for automated or human-expert supported analyses and interpretation of observations. Image pattern recognition algorithms will be also employed to scan predictive modelling databases, to identify ocean/atmospheric patterns and the initiation of extreme incidents (hurricanes, medicanes), issuing early warnings. We will also use other types of data, such as spatial-temporal remote sensing data, blended with social media and citizen science data, to enable continuous dynamic monitoring of rapid changes at the sea surface at a fine spatial resolution and to validate socio-economic impacts. By using spatio-temporal fusion approaches to create fine spatial-temporal resolution images we will achieve spectral-spatial-temporal fusion for coastal shallow environments and reconstruct 10 m spectral images (radar and multispectral) using Sentinel 1-3 images. A possible outcome is a trained network able to reconstruct both spatial and spectral features at the best temporal resolution (<1.5 days). In the same fashion, other pattern recognition services using, for example, climate projection datasets, can be made available as decision support services for long-term planning and development of mitigation strategies. The automated recognition of specific patterns of interest, will facilitate the development of services and applications that operate in near real-time.

• Duration:  M13 - M32
• Task Lead:  Dimokritio Panepistimio Thrakis (DUTH)

This task deals with the extraction of analytics tailored to the user’s needs from the services provided in tasks 5.2 to 5.5, using the service layer established in task 5.1 to satisfy requirements elicited in WP1. This task will also provide access to an Application Integration Environment that supports researchers to adapt and co-designed their analytics applications for integration in a Cloud platform. Supported by dedicated guidelines prepared for that purpose by NextGEOSS and ESA Thematic Exploitation Platforms, the service developers are able to evolve their services, and receive support for application integration and their packaging as appliances in Cloud-agnostic way that is ready to be deployed into different Cloud platforms (e.g., EGI, DIAS, Commercial providers) tapping computational and data resources. This task will provide access to a user workspace persistently and transparently managed to achieve infrastructure scalability taking advantage of the JupyterHub over a Kubernetes Cluster possibly deployed in existing Copernicus DIAS. For co-design and stakeholder engagement, this task offers an online co-creation space with intuitive UIX for added-value product development. It includes a plethora of algorithms, which can be combined with heterogeneous data providers and consumers with the aid of a visual workflow builder. This work inherits its basis from the Copernicus App Lab, a H2020 funded project that elaborated and validated possible evolutions of the Copernicus Data Services to further empower downstream service providers. As an example, an aquaculture farmer might use a service from task 5.4 to model the optimal ration size and fish weight in offshore farm aquaculture depending on water parameters, such as its temperature. Another example may be a cargo ship with a new, changing destination port that has to (re)apply for clearance and port permission using the DTO on spatial ballast water “same area” risk assessment for invasive species modelling. In addition, users may define a conjugation of factors (e.g., a conjunction of thresholds for a specific area, using a determined model) that triggers an event when these conditions are met. A message broker will be implemented so that subscribing systems can be notified and consume the triggered events.

• Duration:  M13 - M36
• Task Lead:  INESC TEC - Instituto de Engenharia de Sistemas e Computadores, Tecnologia e Ciência (INESC)