Data on building properties are essential for a variety of urban applications, yet such information remains scarce in many parts of the world. Recent efforts have leveraged instruments such as machine learning (ML), computer vision (CV), and graph neural networks (GNNs) to assess these properties at scale by leveraging urban features or visual information.
However, extracting holistic representations to infer building attributes from multi-modal data across multiple spatial scales and vertical building characteristics remains a significant challenge.
To bridge this gap, we present a innovative framework, that captures both hierarchical urban features and cross-view visual information through a heterogeneous graph. First, we construct a heterogeneous graph that incorporates multi-dimensional urban elements --- buildings, streets, intersections, and urban plots --- to comprehensively represent multi-scale geospatial features.
Constructing heterogeneous urban graph with four element types (buildings, street segments, intersections, and urban plots).
Second, we automatically crop images of individual buildings from both very high-resolution satellite and street-level imagery, and introduce feature propagation on semantic similarity graphs to supplement missing facade information. Third, feature fusion is applied to integrate both morphological and visual features, with holistic representations generated for building attribute prediction.
Generating holistic building representations to support building attribute prediction.Illustration of the heterogeneous GraphSAGE framework for building attribute prediction.
Systematic experiments across three global cities demonstrate that our method outperforms existing CV, ML, and homogeneous GNN-based models, achieving classification accuracies of 86% to 96% across 10 to 12 distinct building types, with mean F1 scores ranging from 0.70 to 0.73. The framework demonstrates robustness to class imbalance and produces more distinctive embeddings for ambiguous categories. In additional task of inferring building age, the method delivers similarly strong performance.
Model performance.
This framework advances scalable approaches for filling gaps in building attribute data and offers new insights into modeling holistic urban environments.
Related Publications
Heterogeneous graph neural networks for building attribute prediction from hierarchical urban features and cross-view imagery
Xiucheng Liang, Winston Yap, Filip Biljecki
Published in ISPRS Journal of Photogrammetry and Remote Sensing, 2026
@article{liang2026heterognn,
title = {Heterogeneous Graph Neural Networks for Building Attribute Prediction from Hierarchical Urban Features and Cross-View Imagery},
author = {Liang, Xiucheng and Yap, Winston and Biljecki, Filip},
year = 2026,
journal = {ISPRS Journal of Photogrammetry and Remote Sensing},
volume = {234},
pages = {185--204},
issn = {0924-2716},
}
We introduce an open framework leveraging multimodal data to enrich building profiles with objective attributes and semantic descriptors. It automates facade detection, reprojects panoramic views, and predicts multi-attribute data.
This study leverages street view imagery and vision-language models to analyze 48,752 building images in Hong Kong, identifying eight building clusters. It demonstrates the potential of scalable SVI-based analyses to capture urban spatial and semantic details, enhancing the understanding of the built environment.
This study explores how building appearances shape urban perception, using machine learning and survey data to analyze human responses to over 250,000 building images from Singapore, San Francisco, and Amsterdam. Findings reveal how architectural styles influence streetscape perceptions, offering insights for architects and city planners.
