Using the Gateway to Build Dashboards and Applications

Gateway APIs can be used from any client that can make HTTPS calls and can be used by portals and applications. In this tutorial you will build a simple Web application exclusively using Gateway API calls that run queries and pipelines you will build using JSON Studio.

The resulting application will be a simple Web page as shown below:

_images/embedding1.jpg

You enter your credentials with which you can query data and then select a state from the drop down. The page uses the credentials and the state code to issue two API calls to the Gateway - one to get the graph and one to get the list of zip codes for that state and present the JSON data.

The steps to implement such a page are:

  • Build the query that returns the list of zip codes per state using the Finder.
  • Test the API through the Gateway.
  • Build the aggregation pipeline that generates all cities in the state and their total population using the Aggregation Builder.
  • Build the graph displaying a scatter plot for cities/towns in the chosen state.
  • Test your APIs through the Gateways.
  • Build your Web page / dashboard.

If you want to follow along with this tutorial you will need zip code data. You can download this data from http://jsonstudio.com/resources/ and create a collection called aZips in your database.

Step 1 & 2 - Use the Finder to build your query

Once you have your zip code data in the aZips collection login to JSON Studio’s Finder and build a query as shown below. Note that you are using a bind variable in the match/where area so that you can pass in the state through the API. Name and save your query as zipList.

_images/embedding2.jpg

Test your query using the generated API - e.g. from a browser:

https://<gateway
host>:8443/Gateway?col=aZips&name=zipList&type=find&prettyPrint&username=<your
username>&pwd=<your password>&db=<your database name>&sdb=<your studio db
name>&host=<mongodb host>&bind.state=%22NY%22

Note that if you run this from the same browser window it will log you out of your JSON Studio session.

Step 3 - Use the Aggregation Builder to build your aggregation pipeline

Login to JSON Studio and navigate to the Aggregation Builder application.

Select the aZips collection and enter a name for a new pipeline named zip3. Then add four stages to the pipeline as follows:

  • A filter stage to match on the state passed in as a bind variable
  • A projection stage that selects the fields and unwinds the location array
  • A grouping stage that groups by city and state and counts number of zip codes, sums the population and picks one of the x/y coordinates.
  • A sorting stage that sorts by population

You can copy/paste the following definition and import the pipeline if you do not want to build it manually:

{ "pipeline_name" : "zip3" ,
"stages" : [
{ "name" : "1400799354423" , "desc" : "select state" , "type" : "filter" , "fields" : [ { "path" : "\"state\" : \"$$state\""}]} ,
{ "name" : "1387030685578" , "desc" : "flat" , "type" : "fields" , "project" : [ { "path" : "city" , "omit" : false} , { "path" : "pop" , "omit" : false} , { "path" : "state" , "omit" : false} , { "path" : "loc" , "omit" : false}] , "unwind" : "loc"} ,
{ "name" : "1387030740025" , "desc" : "group by city/state" , "type" : "group" , "_id" : [ { "path" : "city: \"$city\""} , { "path" : "state: \"$state\""}] , "fields" : [ { "path" : "count : {$sum : 1}"} , { "path" : "_sum_pop : {$sum: \"$pop\"}"} , { "path" : "x: {$min: \"$loc\"}"} , { "path" : "y : {$max: \"$loc\"}"}]} ,
{ "name" : "1387030856626" , "desc" : "sort by pop" , "type" : "sort" , "skip" :  null  , "limit" :  null  , "sort" : [ { "path" : "_sum_pop" , "asc" : false}
]}]}

Run the pipeline to see that it is working. Make sure you have a bind variable defined for state and set it to “CA” for this example. You should get document such as:

{
"_id": { "city": "LOS ANGELES", "state": "CA" },
"count": 112,
"_sum_pop(POPULATION)": 4204590,
"x": -118.473967,
"y": 34.133932
},
{
"_id": { "city": "SAN DIEGO", "state": "CA" },
"count": 68,
"_sum_pop(POPULATION)": 2098596,
"x": -117.243307,
"y": 33.027854
},
...

Each resulting document has the state and city as the identifier, the number of zip codes in that city, the total population in the zip codes in that city and the min x/y coordinates among zip codes.

Step 4 - Use the Visualizer to build a graph

Once your aggregation data looks right, click on the Visualizer link (the light bulb icon) to navigate to the Visualizer.

Build a scatter plot as shown in the image below by dragging/dropping attributes from the left to the input fields or copy paste the attached export definition and import it to create a graph definition called zip2:

{ "ln0" : "" , "ln1" : "" , "ln2" : "" , "ln3" : "" , "ln5" : "" , "sn3" : "x" , "chartType" : "scatterChart" , "manualChartType" : "" , "sn1" : "y" ,
"sn2" : "_sum_pop" , "st1" : "_id.state" , "st2" : "_id.city" , "lt1" : "" , "tt1" : "" , "tt2" : "" , "tn0" : "" , "tt3" : "" , "bn1" : "" , "bn2" : "" ,
"bt1" : "" , "bn3" : "" , "bn4" : "" , "bn5" : "" , "overrideX" : "" , "overrideY" : "" , "legend" : "yes" , "logY" : "no" , "logX" : "no" , "shouldSortLine" : "no" ,
"ln4" : ""}
_images/embedding3.jpg

Step 5 - Test your pipeline and graph through the Gateway

First - test your pipeline through the gateway using a browser and a URL similar to:

https://<gateway host>:8443/Gateway?col=aZips&name=zip3&type=agg&username=<your
username>&pwd=<your password>&db=<your database name>&sdb=<your studio db
name>&host=<your mongodb host>&bind.state=%22RI%22

The output should look like:

{"output": [{ "_id" : { "city" : "CRANSTON" , "state" : "RI"} , "count" : 14 ,
"_sum_pop" : 352808 , "x" : -71.506102 , "y" : 41.826431}, ...

Then, run the same API but add the following to the URL to produce a graph instead of JSON data:

https://<gateway host>:8443/Gateway?col=aZips&name=zip3&type=agg&username=<your
username>&pwd=<your password>&db=<your database name>&sdb=<your studio db
name>&host=<your mongodb host>&bind.state=%22RI%22&output=graph.zip2

You should get a graph similar to the following image:

_images/embedding4.jpg

Now everything is working and you’re ready to build a page that uses the APIs.

Step 6 - Build your Web page

In a real environment you would use your portal or application framework to make Gateway HTTPS calls. In this tutorial you will build a simple static HTML page that has two input fields (for entering credentials), two iframes that will be used to display the results, and a drop down listing all states. When you select a state, a Javascript function is called. This function loads each of the iframes with a Gateway URL that you used before for testing. The entire Web page code is very simple and looks like:

<html>
<head>
<script>
function selectedState(stateName) {
   if (stateName.length > 0) {
      var u = document.getElementById('username').value;
      var p = document.getElementById('pwd').value;
      chartURL = "https://localhost:8443/Gateway?col=aZips&name=zip3&type=agg&username=" + u + "&pwd=" + p + "&db=tweets&sdb=lmrm&host=localhost&bind.state=%22" + stateName + "%22&output=graph.zip2&graphPoints=2000&keepSession=1";
      listURL = "https://localhost:8443/Gateway?col=aZips&name=zipList&type=find&prettyPrint&username=" + u + "&pwd=" + p + "&db=tweets&sdb=lmrm&host=localhost&bind.state=%22" + stateName + "%22&keepSession=1";
      var el = document.getElementById('chart');
      el.src = chartURL;
      var el2 = document.getElementById('list');
      el2.src = listURL;
   }
}
</script>
</head>
<body>
Username: <input type="text" name="username" id="username"/>
Password: <input type="password" name="pwd" id="pwd"/>
<select id="state" onchange="selectedState(this.value)">
   <option value="">Select State</option>
   <option value="AL">Alabama</option>
   <option value="AK">Alaska</option>
   <option value="AZ">Arizona</option>
   <option value="AR">Arkansas</option>
   <option value="CA">California</option>
   <option value="CO">Colorado</option>
   <option value="CT">Connecticut</option>
   <option value="DE">Delaware</option>
   <option value="DC">District Of Columbia</option>
   <option value="FL">Florida</option>
   <option value="GA">Georgia</option>
   <option value="HI">Hawaii</option>
   <option value="ID">Idaho</option>
   <option value="IL">Illinois</option>
   <option value="IN">Indiana</option>
   <option value="IA">Iowa</option>
   <option value="KS">Kansas</option>
   <option value="KY">Kentucky</option>
   <option value="LA">Louisiana</option>
   <option value="ME">Maine</option>
   <option value="MD">Maryland</option>
   <option value="MA">Massachusetts</option>
   <option value="MI">Michigan</option>
   <option value="MN">Minnesota</option>
   <option value="MS">Mississippi</option>
   <option value="MO">Missouri</option>
   <option value="MT">Montana</option>
   <option value="NE">Nebraska</option>
   <option value="NV">Nevada</option>
   <option value="NH">New Hampshire</option>
   <option value="NJ">New Jersey</option>
   <option value="NM">New Mexico</option>
   <option value="NY">New York</option>
   <option value="NC">North Carolina</option>
   <option value="ND">North Dakota</option>
   <option value="OH">Ohio</option>
   <option value="OK">Oklahoma</option>
   <option value="OR">Oregon</option>
   <option value="PA">Pennsylvania</option>
   <option value="RI">Rhode Island</option>
   <option value="SC">South Carolina</option>
   <option value="SD">South Dakota</option>
   <option value="TN">Tennessee</option>
   <option value="TX">Texas</option>
   <option value="UT">Utah</option>
   <option value="VT">Vermont</option>
   <option value="VA">Virginia</option>
   <option value="WA">Washington</option>
   <option value="WV">West Virginia</option>
   <option value="WI">Wisconsin</option>
   <option value="WY">Wyoming</option>
</select>
<br/>
<iframe id="chart" src="http://jsonstudio.com" width="1000" height="800"></iframe>
<iframe id="list" src="http://jsonstudio.com" width="400" height="800"></iframe>
</body>
</html>

Each time you select a state from the drop down the two Gateway calls will be made and the iframes populated with data - one with the graph and one with the JSON data.

There are two important things you will need to do for this Web page (and embedding in your application) to work:

1. As explained in Configuring for Embedded URLs in Dashboards and Applications, you may need to configure the Gateway server to allow for such frame embedding from your application.

2. Notice that the URLs used (and thus the API calls) are a little different than those made when you tested you API. Specifically, keepSession=1 was added. If you only make a single API call at a time then you do not need this attribute. But if you make multiple calls from the same session (e.g. the same browser session) and cannot control timing, and you do not add this attribute, then each API call will invalidate the session when it is finished. This creates a race condition and some APIs will fail prematurely (because their session data will be removed). Adding this attribute ensures that the invocation does not invalidate the session. When doing this you may have to manually invalidate the session when all calls are complete or let the server reclaim the session after 30 minutes. Also, when using this feature you only need to provide credentials once and not on each gateway call.

When you have many calls in a dashboard that are made in parallel, the calls that belong to the same session are synchronized. Therefore, you do not need to manage the timing and order for most such URLs. However, both charts and reports have a 2-step process where the data is prepared after-which a redirect to the rendering page occurs to use the data. This redirect needs to be completely before the next request comes in or it may use the previous data. If your pieplines or finds take a bit of time (a couple of seconds) then this is enough and you will not experience a race condition. Otherwise, consider adding a delay to your embedding code such as:

setTimeout( function() {
   listURL = "<your api url>";
   var el2 = document.getElementById('chart3');
   el2.src = listURL;
 }, 3000 );

Table Of Contents

Previous topic

Using the Gateway to build composites (Chaining API calls)

Next topic

Implementing a Simple Asset Management System

Copyright © 2013-2016 jSonar, Inc
MongoDB is a registered trademark of MongoDB Inc. Excel is a trademark of Microsoft Inc. JSON Studio is a registered trademark of jSonar Inc. All trademarks and service marks are the property of their respective owners.