The Urban Center for Computation and Data (UrbanCCD) is a research initiative of Argonne National Laboratory and the Computation Institute of the University of Chicago. We create computational tools to better understand cities. One of these is Plenario, our hub for open geospatial data. PostGIS makes the spatial operations at the heart of Plenario possible.
Over the last four years, government agencies have published more and more of their data on open data portals. However, someone who wants to survey all of the data available for their city might have to dig through the websites of their city, county, state, and federal governments to bring together all the datasets they need.
Plenario collects datasets from disparate sources and lets users find all data available for their area of interest with a single spatial query. In addition to aiding data discovery, Plenario can offload some data preparation work by applying filters and aggregates to data before delivering it to the user.
Ingestion: Users submit links to spatial datasets they want to see in Plenario. Those can be CSVs with events that have a location and time column (think crimes or building permits) or ESRI shapefiles (think neighborhood boundaries or bike lanes). Using tools from the GDAL suite, we ETL these datasets into our postgres database with a common spatial schema so that the datasets are ready to query.
Query: A Python web application translates incoming API calls into SQL queries using SQLAlchemy (and its GeoAlchemy extension). Discovery queries accept spatial ranges formatted as GeoJSON to enable queries like “Please show me all available datasets for the South Side of Chicago between 2012 and 2014.” Other queries combine preloaded datasets to enable queries like “Give me all health inspection complaints filed in police district 12” or “Give me a count of filming permits per neighborhood of Chicago in the last year.”
In addition to making Plenario technically possible, the free and open governance of PostGIS makes it an operational fit. All of the tools we build at UrbanCCD are free and open source. A vital part of our mission is to allow governments and citizens to use the tools we build at no or low cost in their own cities. Because PostGIS is free as in beer, it doesn’t add licensing costs to cash-strapped cities. Because PostGIS is free as in speech, it fits our goals for greater transparency in government and science. But that does not imply a compromise in technical quality. We can rely on PostGIS and successful client applications like the City of Chicago’s OpenGrid can rely on us.
As a software engineer at the Howard Hughes Medical Institute, I work on a collaborative neuron reconstruction and analysis software called CATMAID 1 (screenshot: 3), which is used for neuroscience research. We use PostGIS to represent neurons in a 3D space.
They consist of 3D points that reference their parent nodes or are the root [=soma of neuron] if they have no parent). Together with synapses, point clouds and TIN meshes for modeling compartments in a dataset, they model the spatial aspects of our neuroscience world. Users create those neuron reconstructions manually in a collaborative fashion plus segmentation programs can be used as additional data source. Using its spatial indices, PostGIS helps us to quickly query neurons in a particular field of view. The space of a single project contains sometimes 100s of millions of interconnected individual points. We also do bounding box intersection queries between neurons and compartment meshes, which then refine in the front-end by doing more precise intersection tests.
This software is used by quite a few research labs and as far as I know they all do their own hosting with a dedicated server and this is what we do as well. The reason being mainly that wth larger datasets, we benefit from machines with a lot of RAM (>256G), fast SSD/NVMe drives and many CPUs as well as fast local data access for e.g. image data.
Thanks so much for making PostGIS work well in non-GIS contexts too—-it makes my life much easier!
Vanguard Appraisals is new to the GIS world. In fact, we aren’t really in the GIS world; we just kind of brush up against it. We do mass property appraisal for entire county and city jurisdictions, and we develop software to collect, price and maintain values. We also host assessment data online so that homeowners can search and find property information much simpler from the comfort of their own home. Our software and websites are used in 7 states (IA, IL, MN, MO, NE, ND, SD).
Nautilytics is a small data visualization and GIS startup based out of Boston, MA. We use PostGIS and PostgreSQL, among other open-source tools to build powerful web applications for US government organizations, public, and private sector companies.