Real-time Data Enrichment to Esri Feature Layer
/
Adding Information When Information Can’t be Added
This challenge presented itself months ago as we begin to build a new application using Esri®’s Tracker for ArcGIS® (Tracker) – a mobile workforce tracking app that enables organizations to know the location of field personnel and monitor where they have been. We wanted to provide information not already available through Tracker, such as contact information, in real time alongside the Tracker locational data.
Problems with Schema
There was a problem with adding static information to the feature service, Esri’s web service on ArcGIS – a schema change is not possible within the spatiotemporal big data store. A schema defines the physical structure of the geodatabase along with the rules, relationships, and properties of each dataset in the geodatabase. In the case of the spatiotemporal big data store, the schema is locked once it is created.
Adding real-time information for each of the contacts would have to be done outside the standard Esri product configuration and tool set. We would need to acquire responses from the users, add the new column schema identifying the contact information, and populate it when the layer is requested by the Esri ArcGIS Online/Portal web map. All this needed to be done without changing the schema within the spatiotemporal big data store or impacting the performance of fetching the information by the end user.
GeoQ and Koop to the Rescue
Combining two great products solved our problem – GeoDecisions GeoQ, a Platform as a Service (PaaS) product, along with Koop, an Esri tool, the swiss army knife for difficult problems. Koop is an open source product from Esri that allows one to standup a feature server - permitting the injection of code anywhere in the lifecycle of a feature layer request. GeoQ is the web-based hub where data from multiple other sources can be added.
By using Koop, we create a feature layer for the user. The user requests the information from the Koop layer. Koop then connects to a client’s ArcGIS Online or Portal for ArcGIS feature service item backed by the spatiotemporal big data store and pulls the data for GeoQ to process. Within GeoQ, columns are added to the original schema to provide the new data being added to the response, such as individual’s name, number, and email. Koop takes the schema with the additional columns and new data, formats it nicely, and serves it up to the original requester without ever changing the source layer in the spatiotemporal big data store.
We have used Koop and GeoQ in the past to create highly cached layers that would handle tens of millions of requests a day without causing any strain on our databases and other infrastructure. With the extensive number of products and features that the Esri stack provides consumers and developers, it is very exciting to see what can be built. Koop and GeoQ can add in countless data sources to your solutions without needing to maintain the data. These tools are the perfect glue to tie these things together in ways that would not have previously been possible.
