Skip to content

StreamPipes

StreamPipes Logo

Within the BigGIS project, we added several GIS-specific extensions to the StreamPipes platform.

With StreamPipes, it is possible to build distributed pipelines to process your data in real-time without development effort by providing an easy-to-use graphical user interface on top of existing stream processing frameworks. Pipelines consist of data streams, real-time event processors and data sinks. StreamPipes provides a system that can be used by domain experts with little to no technical understanding to explore and analyse their data. The whole system is designed in a way that it is flexible and can be easily extended by adding data streams, data processors or data sinks. A community edition of StreamPipes is available under the Apache Licence.

Processing of climate data

SenseBoxAdapter

This component provides the semantic description for the externally created SenseBox data stream for the integration into StreamPipes

datatype name URI in RDF
timestamp createdAt
string boxId http://schema.org/id

SenseboxMetadataEnricher

This component enriches each message in the SenseBox measurements data stream with meta-data (location, OpenSenseMap-Id)

Raster data processing

This is achieved throuh geotrellis library... TODO

RasterDataEndlessSource

This component generates an endless Kafka stream of rasterdata messages to easily test and debug other rasterdata processing components

RasterDataAdapter

This component provides the semantic description for the RasterDataEndlessSource

TODO: RasterDataProcessingCompoent