JCEEES
Journal of Computer & Electrical and Electronics Engineering Sciences

JCEEES aims to publish original articles covering the theoretical foundations of major computer, electronic and electrical engineering sciences, as well as academic, commercial and educational aspects that propose new ideas for the application and design of artificial intelligence, software and information systems. In addition to wide-ranging regular topics, JCEEES also makes it a principle to include special topics covering specific topics in all areas of interest mainly in computational medicine, artificial intelligence, computer science, and electrical & electronic engineering science.

eISSN: 2980-177X
Creative Commons CC BY-NC-ND 4.0 International License.

Online Manuscript System

Index
Original Article
An integrated deep learning approach for 3D urban scene generation using monocular satellite imagery
Abstract
Aims: The aim of this study is to build a pipeline leveraging deep learning for 3D reconstruction of urban environments using only single high-resolution satellite images, while comparing strengths and weaknesses of the proposed system with its alternatives like multi-view or LiDAR-based methods.
Methods: We proposed a pipeline system consisting of several steps with different deep learning models. Each step works independently to complete different tasks. Firstly, the system generates Digital Surface Model (DSM) using a Pix2Pix conditional GAN with perceptual loss, then a Mask R-CNN model is used to build segmentation and finally a ResNet-50 model classifies roof type of buildings. Models were trained on SpaceNet Urban 3D, Inria Aerial Labeling, and GATE roof datasets, and MSE, IoU/mAP and Accuracy/F1 scores are preferred as evaluation metrics for elevation accuracy, segmentation quality and classification performance, respectively.
Results: DSM generation achieved MSE of 986.70 (mean absolute deviation of 31 gray levels), with 90% of pixels maintaining errors below 50 levels. Instance segmentation attained 0.712 mAP@IoU50, declining to 0.358 at stricter IoU75 thresholds, revealing boundary precision challenges. Roof classification reached 80% accuracy with strong performance on common types (hip roofs: F1=0.91). The system synthesizes these outputs into exportable 3D scenes combining terrain meshes, extruded building geometries, and vegetation elements.
Conclusion: This work validates that monocular satellite imagery can be sufficiently used as a sole input for a system using different deep learning models to generate functionally useful 3D urban models. The proposed system offers a practical solution for rapid urban analysis when additional data like multi-view image or laser scan is unavailable. The coordinated integration of multiple deep learning components yields promising results for urban reconstruction using conventional satellite imagery while geometric precision remains below production-grade standards.

Keywords : Digital surface model, pix2pix, mask r-cnn, remote sensing

Download Article


1. Buyukdemircioglu, M., Can, R., & Kocaman, S. (2021). Deep learning based roof type classification using very high resolution aerial imagery. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, XLIII-B3-2021, 55-60. https://doi.org/10.5194/isprs-archives-XLIII-B3-2021-55-2021
2. Chen, S., Shi, Y., Xiong, Z., & Zhu, X. X. (2023). HTC-DC Net: monocular height estimation from single remote sensing images. IEEE Transactions on Geoscience and Remote Sensing, 61, 1-18. https://doi.org/10.1109/TGRS.2023.3321255
3. Ghamisi, P., & Yokoya, N. (2018). IMG2DSM: height simulation from single imagery using conditional generative adversarial net. IEEE Geoscience and Remote Sensing Letters, 15(5), 794-798. https://doi.org/10.1109/LGRS.2018.2806945
4. Goldberg, H., Brown, M., & Wang, S. (2017). A benchmark for building footprint classification using orthorectified RGB imagery and digital surface models from commercial satellites. In 2017 IEEE Applied Imagery Pattern Recognition Workshop (AIPR) (pp. 1-7). IEEE. https://doi.org/10.1109/AIPR.2017.8457973
5. He, K., Gkioxari, G., Dollar, P., & Girshick, R. (2020). Mask R-CNN. IEEE Transactions on Pattern Analysis and Machine Intelligence, 42(2), 386-397. https://doi.org/10.1109/TPAMI.2018.2844175
6. He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 770-778). IEEE. https://doi.org/10.1109/CVPR.2016.90
7. Hou, T., & Li, J. (2024). Application of mask R-CNN for building detection in UAV remote sensing images. Heliyon, 10(19), e38141. https://doi.org/10.1016/j.heliyon.2024.e38141
8. Hristov, E., Petrova-Antonova, D., Petrov, A., Borukova, M., & Shirinyan, E. (2023). Imagery dataset for rooftop detection and classification. Zenodo. https://doi.org/10.5281/zenodo.7633595
9. Isola, P., Zhu, J. Y., Zhou, T., & Efros, A. A. (2017). Image-to-image translation with conditional adversarial networks. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 5967-5976). https://doi.org/10.1109/CVPR.2017.632
10. Kong, L., Qian, H., Wu, Y., Niu, X., Wang, D., & Huang, Z. (2023). Simplification and regularization algorithm for right-angled polygon building outlines with jagged edges. ISPRS International Journal of Geo-Information, 12(12), 469. https://doi.org/10.3390/ijgi12120469
11. Li, W., Yang, H., Hu, Z., Zheng, J., Xia, G. S., & He, C. (2024). 3D building reconstruction from monocular remote sensing images with multi-level supervisions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 27728-27737). IEEE. https://doi.org/10.1109/CVPR.2024.02702
12. Liu, C.-J., Krylov, V. A., Kane, P., Kavanagh, G., & Dahyot, R. (2020). IM2ELEVATION: building height estimation from single-view aerial imagery. Remote Sensing, 12(17), 2719. https://doi.org/10.3390/rs12172719
13. Maggiori, E., Tarabalka, Y., Charpiat, G., & Alliez, P. (2017). Can semantic labeling methods generalize to any city? The Inria aerial image labeling benchmark. In 2017 IEEE International Geoscience and Remote Sensing Symposium (IGARSS) (pp. 3226-3229). IEEE. https://doi.org/10.1109/IGARSS.2017.8127684
14. Ölçer, N., Ölçer, D., & Sümer, E. (2023). Roof type classification with innovative machine learning approaches. PeerJ Computer Science, 9, e1217. https://doi.org/10.7717/peerj-cs.1217
15. Panagiotou, E., Chochlakis, G., Grammatikopoulos, L., & Charou, E. (2020). Generating elevation surface from a single RGB remotely sensed image using deep learning. Remote Sensing, 12(12), 2002. https://doi.org/10.3390/rs12122002
16. Ramadhani, A., & Alya, N. F. (2024). Building extraction from unmanned aerial vehicle imagery using mask-RCNN (case study: Institut Teknologi Sepuluh Nopember, Surabaya). E3S Web of Conferences, 600, 06003. https://doi.org/10.1051/e3sconf/202460006003
17. Ronneberger, O., Fischer, P., & Brox, T. (2015). U-Net: convolutional networks for biomedical image segmentation. In N. Navab, J. Hornegger, W. M. Wells, & A. F. Frangi (Eds.), Medical image computing and computer-assisted intervention-MICCAI 2015 (Lecture Notes in Computer Science, Vol. 9351, pp. 234-241). Springer. https://doi.org/10.1007/978-3-319-24574-4_28
18. Shailja, Yadava, M., Dwivedi, R., & Pardeshi, P. J. (2025). Classifying roof types from orthophotos: a Swin Transformer-based approach. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, X-5/W2-2025, 577-584. https://doi.org/10.5194/isprs-annals-X-5-W2-2025-577-2025
19. Surya. (2024, July). Roof-Type-Classifier dataset [Data set]. Roboflow Universe. https://universe.roboflow.com/surya-dmwfw/roof-type-classifier-bafod
20. Tarabalka, Y., Girard, N., Tripodi, S., Larrosa, C., Bauchet, J. P., Fonteix, G., ..., & Madelain, V. (2022). Reconstruction of 3D environments from satellite images by AI and computational geometry for exploitation in mixed reality. STO-MP-MSG-197. NATO Science and Technology Organization.
21. Vincent, J. M., & Varalakshmi, P. (2024). Extraction of building footprint using mask-RCNN for high resolution aerial imagery. Environmental Research Communications, 6(7), 075015. https://doi.org/10.1088/2515-7620/ad5b3d
22. Wei, S., Zhang, T., Ji, S., Luo, M., & Gong, J. (2023). BuildMapper: a fully learnable framework for vectorized building contour extraction. ISPRS Journal of Photogrammetry and Remote Sensing, 197, 87-104. https://doi.org/10.1016/j.isprsjprs.2023.01.015
23. Weinstein, B. G., Marconi, S., Aubry-Kientz, M., Vincent, G., Senyondo, H., & White, E. P. (2020). DeepForest: a Python package for RGB deep learning tree crown delineation. Methods in Ecology and Evolution, 11(12), 1743-1751. https://doi.org/10.1111/2041-210X.13472
24. Weinstein, B. G., Marconi, S., Bohlman, S. A., Zare, A., & White, E. P. (2019). Individual tree-crown detection in RGB imagery using semi-supervised deep learning neural networks. Remote Sensing, 11(11), 1309. https://doi.org/10.3390/rs11111309
25. Zhang, W., Qi, J., Wan, P., Wang, H., Xie, D., …, & Yan, G. (2016). An easy-to-use airborne LiDAR data filtering method based on cloth simulation. Remote Sensing, 8(6), 501. https://doi.org/10.3390/rs8060501
26. Zhao, K., Kang, J., Jung, J., & Sohn, G. (2018). Building extraction from satellite images using mask R-CNN with building boundary regularization. In 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) (pp. 242-2424). IEEE. https://doi.org/10.1109/CVPRW.2018.00045
Article available in :
Volume 4, Issue 2, 2026
Page : 56-63
https://doi.org/10.51271/JCEEES-0037
Article Information
Received : 09 Ara 2025
Accepted : 27 Mar 2026
Published : 13 Eki 2026
Corresponding Author :

Ömer Can Karadağ: Department of Software Engineering, Faculty of Engineering and Architecture, İstanbul Gelişim University, İstanbul, Turkiye

[email protected]
Citation : Karadağ, Ö. C., & Altuntaş, N. (2026). An integrated deep learning approach for 3D urban scene generation using monocular satellite imagery. J Comp Electr Electron Eng Sci, 4(2), 56-63. https://doi.org/10.51271/JCEEES-0037

Read: 6
Download : 4
Cited: 0
_Footer