Spatio-B-RAG

Project Description

The Spatio-B-RAG project focuses on developing a Retrieval-Augmented Generator (RAG) that leverages domain-specific spatial knowledge to enable predictive damage assessments for bridge structures. By linking heterogeneous, multimodal data such as inspection reports, construction plans, photographic documentation, and point clouds within a spatial context, a dynamic knowledge system is created. The foundation lies in spatially annotated knowledge graphs developed as part of the RaumLink project, which utilized the ReLoc ontology to structure semi-structured bridge inspection data from the SIB-Bauwerke database based on the “Straßeninformationsbank” directive, subsystem structural data (ASB-ING). These were tested on a reference bridge in Worms and serve as training data for domain-specific RAGs for structural models. The RAG will be developed using symbolic and sub-symbolic methods, including Named Entity Recognition (NER) with BERT, multimodal models such as VisualBERT and LXMERT, and Graph Neural Networks (GNNs), to process text, image, and 3D data effectively. The integration of spatial relationships and domain-specific ontologies allows for efficient information retrieval, serving as a foundation for predictions and maintenance decisions.