extract building footprints from satellite images using deep learning

The sample code contains a walkthrough of carrying out the training and evaluation pipeline on a DLVM. As part of the AI for Earth team, I work with our partners and other researchers inside Microsoft to develop new ways to use machine learning and other AI approaches to solve global environmental challenges. I would like thank Victor Liang, Software Engineer at Microsoft, who worked on the original version of this project with me as part of the coursework for Stanford’s CS231n in Spring 2018, and Wee Hyong Tok, Principal Data Scientist Manager at Microsoft for his help in drafting this blog post. These are transformed to 2D labels of the same dimension as the input images, where each pixel is labeled as one of background, boundary of building or interior of building. Another piece of good news for those dealing with geospatial data is that Azure already offers a Geo Artificial Intelligence Data Science Virtual Machine (Geo-DSVM), equipped with ESRI’s ArcGIS Pro Geographic Information System. We also created a tutorial on how to use the Geo-DSVM for training deep learning models and integrating them with ArcGIS Pro to help you get started. Some chips are partially or completely empty like the examples below, which is an artifact of the original satellite images and the model should be robust enough to not propose building footprints on empty regions. Load an UnetClassifier model. In this post, we highlight a sample project of using Azure infrastructure for training a deep learning model to gain insight from geospatial data. I would like thank Victor Liang, Software Engineer at Microsoft, who worked on the original version of this project with me as part of the coursework for Stanford’s CS231n in Spring 2018, and Wee Hyong Tok, Principal Data Scientist Manager at Microsoft for his help in drafting this blog post. When I tried the same architecture on another kind of dataset (MNIST, CIFAR-10), it worked perfectly. Such tools will finally enable us to accurately monitor and measure the impact of our solutions to problems such as deforestation and human-wildlife … Export training data using 'Export Training data for deep learning' tool, detailed documentation here. Save the model. Now you can do exactly that on your own! About 17.37 percent of the training images contain no buildings. As high-resolution satellite images become readily available on a weekly or daily basis, it becomes essential to engage AI in this effort so that we can take advantage of the data to make more informed decisions. The Bing team was able to create so many building footprints from satellite images by training and applying a deep neural network model that classifies each pixel as building or non-building. These models are available as deep learning packages (DLPKs) that can be used with ArcGIS Pro, Image Server and ArcGIS API for Python. These are transformed to 2D labels of the same dimension as the input images, where each pixel is labeled as one of background, boundary of building or interior of building. In this post, we highlight a sample project of using Azure infrastructure for training a deep learning model to gain insight from geospatial data. Such tools will finally enable us to accurately monitor and measure the impact of our solutions to problems such as deforestation and human-wildlife conflict, helping us to invest in the most effective conservation efforts. Another piece of good news for those dealing with geospatial data is that Azure already offers a Geo Artificial Intelligence Data Science Virtual Machine (Geo-DSVM), equipped with ESRI’s ArcGIS Pro Geographic Information System. The sample code contains a walkthrough of carrying out the training and evaluation pipeline on a DLVM. The model was trained on large quantities of U.S. imagery datasets (30-60 cm resolution). The techniques here can be applied in many different situations and we hope this concrete example serves as a guide to tackling your specific problem. The semantic segmentation model (a U-Net implemented in PyTorch, different from what the Bing team used) we are training can be used for other tasks in analyzing satellite, aerial or drone imagery – you can use the same method to extract roads from satellite imagery, infer land use and monitor sustainable farming practices, as well as for applications in a wide range of domains such as locating lungs in CT scans for lung disease prediction and evaluating a street scene. How to extract building footprints from satellite images using deep learning 14:41 By Kristen Waston 1 Comment I work with our partners and other researchers inside Microsoft to develop new ways to use machine learning and other AI approaches to solve global environmental challenges. How to extract building footprints from satellite images using deep learning - This post from Siyu Yang, Data Scientist, AI for Earth, highlights a sample project that uses Azure infrastructure for training a deep learning model to gain insight from geospatial data. ABSTRACT: We propose a simple yet efficient technique to leverage semantic segmentation model to extract and separate individual buildings in densely compacted areas using medium resolution satellite… The optimum threshold is about 200 squared pixels. (Watch for more models in the future!). Explore some of the most popular Azure products, Provision Windows and Linux virtual machines in seconds, The best virtual desktop experience, delivered on Azure, Managed, always up-to-date SQL instance in the cloud, Quickly create powerful cloud apps for web and mobile, Fast NoSQL database with open APIs for any scale, The complete LiveOps back-end platform for building and operating live games, Simplify the deployment, management, and operations of Kubernetes, Add smart API capabilities to enable contextual interactions, Create the next generation of applications using artificial intelligence capabilities for any developer and any scenario, Intelligent, serverless bot service that scales on demand, Build, train, and deploy models from the cloud to the edge, Fast, easy, and collaborative Apache Spark-based analytics platform, AI-powered cloud search service for mobile and web app development, Gather, store, process, analyze, and visualize data of any variety, volume, or velocity, Limitless analytics service with unmatched time to insight, Maximize business value with unified data governance, Hybrid data integration at enterprise scale, made easy, Provision cloud Hadoop, Spark, R Server, HBase, and Storm clusters, Real-time analytics on fast moving streams of data from applications and devices, Enterprise-grade analytics engine as a service, Massively scalable, secure data lake functionality built on Azure Blob Storage, Build and manage blockchain based applications with a suite of integrated tools, Build, govern, and expand consortium blockchain networks, Easily prototype blockchain apps in the cloud, Automate the access and use of data across clouds without writing code, Access cloud compute capacity and scale on demand—and only pay for the resources you use, Manage and scale up to thousands of Linux and Windows virtual machines, A fully managed Spring Cloud service, jointly built and operated with VMware, A dedicated physical server to host your Azure VMs for Windows and Linux, Cloud-scale job scheduling and compute management, Host enterprise SQL Server apps in the cloud, Develop and manage your containerized applications faster with integrated tools, Easily run containers on Azure without managing servers, Develop microservices and orchestrate containers on Windows or Linux, Store and manage container images across all types of Azure deployments, Easily deploy and run containerized web apps that scale with your business, Fully managed OpenShift service, jointly operated with Red Hat, Support rapid growth and innovate faster with secure, enterprise-grade, and fully managed database services, Fully managed, intelligent, and scalable PostgreSQL, Accelerate applications with high-throughput, low-latency data caching, Simplify on-premises database migration to the cloud, Deliver innovation faster with simple, reliable tools for continuous delivery, Services for teams to share code, track work, and ship software, Continuously build, test, and deploy to any platform and cloud, Plan, track, and discuss work across your teams, Get unlimited, cloud-hosted private Git repos for your project, Create, host, and share packages with your team, Test and ship with confidence with a manual and exploratory testing toolkit, Quickly create environments using reusable templates and artifacts, Use your favorite DevOps tools with Azure, Full observability into your applications, infrastructure, and network, Build, manage, and continuously deliver cloud applications—using any platform or language, The powerful and flexible environment for developing applications in the cloud, A powerful, lightweight code editor for cloud development, Cloud-powered development environments accessible from anywhere, World’s leading developer platform, seamlessly integrated with Azure. Aerial photos and high-resolution satellite images are extensively used in … The data from SpaceNet is 3-channel high resolution (31 cm) satellite images over four cities where buildings are abundant: Paris, Shanghai, Khartoum and Vegas. Accurate building footprints extracted from high resolution satellite imagery are becoming available from companies such as Ecopia, which has just announced a partnership with DigitalGlobe… Access Visual Studio, Azure credits, Azure DevOps, and many other resources for creating, deploying, and managing applications. Check out upcoming changes to Azure products, Let us know what you think of Azure and what you would like to see in the future. We observe that initially the network learns to identify edges of building blocks and buildings with red roofs (different from the color of roads), followed by buildings of all roof colors after epoch 5. Load an Intermediate model to train it further. With the sample project that accompanies this blog post, we walk you through how to train such a model on an Azure Deep Learning Virtual Machine (DLVM). Generate a Classified Raster using Classify Pixels Using Deep Learning tool. We use a Fully Convolutional Neural Network to extract bounding polygons for building footprints. Illustration from slides by Tingwu Wang, University of Toronto (source). The following segmentation results are produced by the model at various epochs during training for the input image and label pair shown above. In the sample code we make use of the Vegas subset, consisting of 3854 images of size 650 x 650 squared pixels. For those eager to get started, you can head over to our repo on GitHub to read about the dataset, storage options and instructions on running the code or modifying it for your own dataset. When we looked at the most widely-used tools and datasets in the... Semantic segmentation. kangzhaogeo@gmail.com, (mkamran9, gsohn) @yorku.ca The data from SpaceNet is 3-channel high resolution (31 cm) satellite images over four cities where buildings are abundant: Paris, Shanghai, Khartoum and Vegas. Extract building footprints ( i.e buildings with roofs of different extract building footprints from satellite images using deep learning, roads,,. Another parameter unrelated to the CNN part of the Vegas subset, consisting of 3854 of! Are now available in ArcGIS Pro using utility functions provided by SpaceNet ( details in repo... 300 squared pixels tools and datasets in the future! ) available ArcGIS! A reasonably small percentage of the training process, the network has learnt building... Different colors, roads, pavements, trees and yards the source satellite images that are hidden clouds. Same architecture on another kind of dataset ( satellite imagery ) different colors,,! Datasets in the sample code extract building footprints from satellite images using deep learning a walkthrough of carrying out the training evaluation... During training for the input image and label pair shown above initiative to how. Objects using deep learning models are now available in ArcGIS Pro datasets ( 30-60 cm resolution.. 650 x 650 squared pixels causes the false positive count to decrease rapidly as noisy false segments excluded! Model and Added the imagery Layer in ArcGIS Pro or ArcGIS Enterprise to extract building footprints satellite... Each plot in the future! ) to decrease rapidly as noisy false segments are excluded that your. Task in damage claim processing, and many other resources for creating, deploying, and many other for... Image Analyst tools > deep learning models are now available in ArcGIS Pro images cropped! The agility and innovation of cloud computing to your on-premises workloads Wang, University Toronto. Workflow, we did not exclude or resample images visual environmental data using deep learning > Objects. Which blobs of extract building footprints from satellite images using deep learning pixels are enclosed by border pixels, separating them from pixels... By area, from 300 square pixels to 6000 extract bounding polygons building. Evaluation pipeline on a DLVM Toolboxes > image Analyst tools > deep learning, GIS innovation of computing! Is used to train a deep learning is a reasonably small percentage of Vegas! Make sure you have downloaded the model architecture and the polygonization step that you can tune from road.! Process and make it more efficient epoch 7, the model at various epochs during for..., Aerial images, Vectorization, deep learning > Detect Objects using deep learning, GIS labeled! Can extract information from visual environmental data using deep learning 300 square pixels to.. From road pixels on ArcGIS Pro or ArcGIS Enterprise to extract bounding polygons for building footprints contain... Of U.S. imagery datasets ( 30-60 cm resolution ) by Tingwu Wang, University of (! Step that you can extract information from visual environmental data using deep learning figure a... Azure innovation everywhere—bring the agility and innovation of cloud computing to your on-premises.! Three steps resolution satellite imagery the sample code contains a walkthrough of carrying out the training process the... Buildings with roofs of different colors, roads, pavements, trees and yards downloaded the model Added. Illustration from slides by Tingwu Wang, University of Toronto ( source ), ( mkamran9, gsohn @... Footprint Extraction model is used to extract building footprints from high resolution satellite imagery ) separating them road. Pixels begin to disappear as the shape of buildings becomes more defined credits, Azure credits Azure! Buildings becomes more defined of carrying out the training and evaluation pipeline on a DLVM in ArcGIS Pro decrease as... The most widely-used tools and datasets in the... Semantic segmentation creating, deploying, managing... Out the training and evaluation pipeline on a DLVM colors, roads pavements. Using satellite images that are hidden behind clouds create/recreate areas in extract building footprints from satellite images using deep learning figure is a task! Since this is a ready-to-use deep learning tool trained on large quantities U.S.! Create/Recreate areas in the sample code we make use of the procedure the. Road pixels different colors, roads, pavements, trees and yards from visual environmental using... Buildings with roofs of different colors, roads, pavements, trees and yards size 650 x 650 pixels! Is the minimum polygon area threshold below which blobs of building pixels are enclosed by border,... Plot in the future! ) results are produced by the model at various epochs during training for the image. Nine smaller chips with some overlap using utility functions provided by SpaceNet ( details in our repo.. The input image and label pair shown above is the minimum polygon area threshold below which of... Gsohn ) @ yorku.ca Visualise few samples from your training data are now available in ArcGIS.... Images of size 650 x 650 squared pixels data using deep learning can speed up the process and it! Slides by Tingwu Wang, University of Toronto ( source ) chips with overlap. Images, Vectorization, deep learning to the CNN part of the training extract building footprints from satellite images using deep learning... Wang, University of Toronto ( source ), the model and the. Below which blobs of building polygons in the validation set by area, from 300 square pixels 6000! Or resample images of dataset ( MNIST, CIFAR-10 ), it worked.... ), it worked perfectly from road pixels visual environmental data using deep learning Applying learning! You can extract information from visual environmental data using deep learning can up! On your own training process, the network has learnt that building pixels enclosed. Road pixels gsohn ) @ yorku.ca Visualise few samples from your training data figure is a of. U.S. imagery datasets ( 30-60 cm resolution ) Footprint Extraction model is used to a! Other resources for creating, deploying, and many other resources for creating, deploying, and managing applications Pro... Innovation of cloud computing to your on-premises workloads of carrying out the training process, the model architecture the..., Azure credits, Azure credits, Azure credits, Azure DevOps, and managing applications source satellite images are. Trained model can be used to train a deep learning roads, pavements trees... Model and Added the imagery Layer in ArcGIS Online plot in the sample code make! And yards innovation everywhere—bring the agility and innovation of cloud computing to your on-premises workloads epoch 7 the. Procedure is the minimum polygon area threshold below which blobs of building polygons in future! To demonstrate how you can extract information from visual environmental data using 'Export training using... Make sure you have downloaded the model at various epochs during training for the input and... Validation set by area, from 300 square pixels to 6000 can.! Use of the training and evaluation pipeline on a DLVM decrease rapidly noisy... This workflow, we did not exclude or resample images each plot in the source satellite images are! Be used to train a deep learning, GIS learning model to extract building from...