JSON Studio configuration options

JSON Studio License

You need to have a license file for JSON Studio to connect to the database (evaluation versions come with a limited license already). JSON Studio is licensed per host (on which it is installed) and per the number of concurrent connections to MongoDB it manages.

Get a license file from your jSonar account manager after you provide the hostname where JSON Studio is to be run. Place the jsonar.license file that you receive in $INSTALLATION_DIR/conf. The license file will look like the following:

$ cat jsonar.license
----- BEGIN LICENSE -----
Hostname=ubuntu1204.jsonar.com
Expiration=20131231
Concurrent=20
DebugHash=5f4dcc3b5aa765d61d8327deb882cf99
----- END LICENSE -----
----- BEGIN SIGNATURE -----
MCwCFHkaDoKzdC6ZYTDh
UFFfxx5aMpe2AhQrz4gM
ILdQ5vtGwBXt5StGfqiP
2Q==
----- END SIGNATURE -----

This license is good for the Studio to run on ubuntu1204.jsonar.com, valid until the end of 2013, and for 20 concurrent users.

Preferences

Preferences control the behavior of JSON Studio applications and are shared among the applications. Thus, if you set a value on one preference screen it will also affect the other applications (if that other application uses that preference). Not all applications use all preferences and the Preferences screen per application only shows the values that affect it.

Finder:

  • Facet depth - controls how facet menus are built and to what depth of dot notation items are added to the facet menu.
  • Facet menu max - controls how many samples are added to the facet menus (as values).
  • Max number of chart/table data points - when using Google viz (vs. the Visualizer) for tables and charts, the maximum number of points/rows to be used.
  • Search on any change - when set to 1, any change to a search widget (such as the facets or additional where/select) causes the search/query to be re-submitted without you having to press the execute button. Be VERY careful using this feature if you have large collections and/or your database is slow since it will issue a query as you make any change; this feature works best when queries are fast. Also, if the Studio is accessing your database over a WAN the result set may not have completed when the ajax response returns and therefore it seems like no update was done. In such cases execute the search manually using the Execute button.
  • Pretty-print query - controls whether the generated query shown in the Query tab is formatted for pretty-print.
  • Auto-build metadata - if you select a collection in the Finder that does not have sampling metadata and this is set to 1 then the Studio will auto-compute metadata when you first select the collection (using quick sampling - see Schema Analyzer/Sampler). If set to 0 the Studio will tell you that it cannot populate the facets and give you two buttons to compute metadata explicitly.
  • JSON pretty indentation - Controls the number of spaces to use for JSON pretty print indentation in all collection viewers.
  • Number of documents to return when adding sort condition (added in a limit call).
  • Compute result set size - when you execute a search or a query in the Finder, this parameter controls whether to automatically compute the result set size or not. 0 means not to compute and 2 means to always compute. 1 means to compute it for collections who’s count is less than the document size requiring an index (see below). Be careful when setting this to 2 when you have large collections and/or slow databases since computing the size of the result set could require a collection scan.
  • Aggregate repeating rows - When you open a tabular display from the Finder, depending on what projections you may have selected, there can be repeating rows in the tabular display. If you set this value to 1 then repeating rows will be aggregated with a counter. NOTE: Opening a tabular report with repeating aggregation enabled is much slower than without because of the additional processing involved.
  • Use aliases - whether or not to display aliases
  • Document size that requires an index for find - Running queries that do not include any field that has an index may take a long time and also overload the database. This only happens for large collections. You can set a threshold that controls whether or not a query can run without some index field (or at least a warning issued - see the next parameter). If this is set to -1 it means no threshold
  • Prevent queries without index - specifies what to do when a query without an index is attempted on a collection that is larger than the above threshold. 0 means no prevention and 1 means disallow it. Default is 0.
  • Set type of LHS panel - If set to 0 then the left-hand-side panel shows the collection structure with sample values for fields. If set to 1 then the left-hand-side panel shows JSON data from a cursor. Default is to show metadata. REMOVED IN V2.4 since tree-view has fields/metadata.
  • UUIDs and GUID - Different mongodb drivers interpret type 3 BinData differently. The Studio uses a Java driver so if your uuids are Microsoft GUIDs and you are using C# drivers, set this preference to 1. Note that this setting controls all conversions for $uuid - not just for the finder. Note also that some double conversion will not function correctly. The main scenarios where this happens is when you use two subqueries – for example, a query in the Finder uses a pipeline for a subquery and that pipeline itself has a filter/match stage that has a separate subquery and these subqueries include lists of UUIDs that need to be converted to Microsoft GUIDs. In such a case double conversion occurs and the result is incorrect. In addition to a preference value of 1 (converting all bin data to UUIDs MS style and querying using bind data type 3 MS style), a setting of 2 will convert all bin data to UUID and run queries using bin data 3 without MS conversion and a setting of 3 will convert all bin data to UUID and run queries using bin data 4.
  • Truncate table view strings after # of characters - Large strings ot arrays may cause the HTML page to be huge and some browsers do not render these pages well. Use this preference to truncate long strings or strings representing arrays in the tabular view. -1 means no limit.

Aggregation Builder:

  • Max number of chart/table data points - when using Google viz (vs. the Visualizer) for tables and charts, the maximum number of points/rows to be used. This also controls how many geo objects can be plotted by the Mapper when displaying aggregated data.
  • Max number of column - - when using Google viz (vs. the Visualizer) for tables and charts, the maximum number of columns/data sets to be used.
  • JSON pretty indentation - Controls the number of spaces to use for JSON pretty print indentation in all collection viewers.
  • Auto-build metadata - if you use a match stage and there is no metadata then the Studio will auto-compute the metadata when you first bring up the list to select from.
  • Limit on the number of documents exported through the Gateway API for an aggregation or a find result.
  • Use aliases - whether or not to display aliases
  • Limit size to add explicitly for each pipeline without a need for an explicit stage.
  • Size of result after which to materialize the result in the background.

Spreadsheet Bridge:

  • Max number of columns in mapping - maximum number of columns in the mapping metadata
  • Max number of rows to generate when exporting data to a spreadsheet (up to 1 Million due to Excel restrictions)
  • JSON pretty indentation - Controls the number of spaces to use for JSON pretty print indentation in all collection viewers.
  • Number of documents to return when adding sort condition (added in a limit call).
  • Result set size - when you execute a search or a query in the Finder, this parameter controls whether to automatically compute the result set size or not. Be careful when setting this to 1 when you have large collections and/or slow databases since computing the size of the result set could require a collection scan.
  • Use aliases - whether or not to display aliases
  • Document size that requires an index for find - Running queries that do not include any field that has an index may take a long time and also overload the database. This only happens for large collections. You can set a threshold that controls whether or not a query can run without some index field (or at least a warning issued - see the next parameter). If this is set to -1 it means no threshold
  • Prevent queries without index - specifies what to do when a query without an index is attempted on a collection that is larger than the above threshold. 0 means no prevention and 1 means disallow it. Default is 0.
  • Compute result set size - when you execute a search or a query in the Finder, this parameter controls whether to automatically compute the result set size or not. 0 means not to compute and 2 means to always compute. 1 means to compute it for collections who’s count is less than the document size requiring an index (mentioned above). Be careful when setting this to 2 when you have large collections and/or slow databases since computing the size of the result set could require a collection scan.

Visualizer:

  • Series must match - when set to 1 all data series must have the same size in order to be plotted together
  • Random factor for auto-selection - when set to 0 the algorithm selecting which graph to plot is completely deterministic. You can control the random factor used in selections.
  • Number of samples to score by auto-generating algorithm - you can control how many graphs are auto-generated for exploratory visualization.
  • Use aliases - whether or not to display aliases

Schema Analyzer:

  • Maximum number of items to retrieve in type query.
  • Maximum runtime for (non-quick sampler). Default is 60 seconds. Up to 20 minutes allowed.
  • Maximal number of samples per field. Default is 50. If you make this number very large rememeber that the GUI builds menus for samples and this may make the GUI slow.
  • Date format to use when casting from strings to dates
  • Use aliases - whether or not to display aliases
  • Document size that requires an index for find - Running queries that do not include any field that has an index may take a long time and also overload the database. This only happens for large collections. You can set a threshold that controls whether or not a query can run without some index field (or at least a warning issued - see the next parameter). If this is set to -1 it means no threshold
  • Prevent queries without index - specifies what to do when a query without an index is attempted on a collection that is larger than the above threshold. 0 means no prevention and 1 means disallow it. Default is 0.

Run-time Viewer:

  • Location of run-time information. 0 means not to persist run-time information. 1 means to store it in the database you connect to in a collection called json_studio_runner_times. 2 means to store it in the Studio database in a collection called lmrm__json_studio_runner_times.
  • Maximum number of run-time values to maintain in-memory by the Run-time Viewer. Default is 1000. Note that the Run-time Viewer will display up to 100 more records than you select. For example, if you set this value to 200 the Run-time Viewer will display up to 300 entries and upon the 301st entry will purge 100 entries.

Mapper:

  • Zoom level to use when centering the map on a selected document.
  • Display type - 0 for satellite/hybrid and 1 for road map.

Administrator Override of Preferences

Preferences are defined per user for themselves. Sometimes you may wish to specify a preference for a user or for all users and not have them be able to modify this setting. An example might be the limit on running non-indexed queries on MongoDB - you may not want users to set this threshold for themselves and might prefer to have this defined centrally.

To do this add a document to a collection named lmrm__preferences_override in the database you intend to use for the Studio database. A document in this collection will look like:

{
   "_id" : ObjectId("55019a5a8ea3e2c884e3d7ec"),
   "app_name" : "JsonStudio",
   "username" : "jane",
   "generateDerivedOidDates" : 0
}

or:

{
   "_id" : ObjectId("55019ab48ea3e2c884e3d7ed"),
   "app_name" : "JsonStudio",
   "username" : "*",
   "generateDerivedOidDates" : 0
}

An asterix will affect the value for all users. Note that when you override a value the user cannot change this value while in JSON Studio and in fact will not even see that preference in the Studio. Note also that it is important to insert the right type into the document or it will not override the user’s setting - for example, since generateDerivedOidDates is an int value, inserting it into lmrm__preferences_override should be done using:

> db.lmrm__preferences_override.insert({"app_name" : "JsonStudio", "username" :
   "jane","generateDerivedOidDates" : NumberInt(0)})
WriteResult({ "nInserted" : 1 })

The values (inserted as strings) and types for which it makes sense to have preference overrides are:

"apiJSONReturnLimit" - int
"revertMatchOp" - int
"defaultAggLimit" - int
"autoCancel" - int
"maxTabularStringSize" - int
"generateDerivedOidDates" - int
"allowDiskUse" - int "allowBuildingIncrementally" - int
"convertGUIDs" - int
"zoom" = int
"aggregateRepeats" - int
"useAliases" - int
"lhsData" - int
"documentSizeRequiresIndex" - int
"warnOrPreventFindWithoutIndex" - int
"runtimeLocation" - int
"displayResultSetCursorSize" - int
"maxSubquerySize" - int
"maxRunnerValues" - int
"limitSort" - int
"maxRuntime" - int
"maxSpreadsheetRows" - int
"maxSpreadsheetCols" - int
"buildMetadata" - int
"maxTypedValues" - long
"directSearch" - int
"prettyPrintGenerated" - int
"maxChartCol" - int
"maxChartRow" - int
"plottedSeriesMustMatch" - int

Read Only Mode

JSON Studio is, for the most part, a querying/analysis/visualization application and most of its operations are read operations. There are writes to the Studio database for profile and saved items (see Studio database) but these are normally saved to a different database than the database where your data resides.

There are exceptions to the rule. For example, the Finder can be used to write results into a database, the Aggregation Builder can have a $out stage, and the Spreadsheet Bridge can be used to import data from a spreadsheet into a collection.

If you do not wish the Studio to write to the database, configure it to run in read-only mode. The installer asks you whether to run in read-only mode. You can perform the change by running the script allow_local_only (.sh or .bat depending on your operating system) or by performing the following manual change:

  • Shutdown the Studio using stop_sonar_finder.sh or stop_sonar_finder.bat depending on your operating system.

  • Edit jsonar/sonarFinder/WEB-INF/web.xml under your installation directory (perhaps make a copy of the file before editing it for recovery purposes).

  • Look for the section:

    <context-param>
       <param-name>jsonStudio.readOnly</param-name>
       <param-value>false</param-value>
    </context-param>
    
  • Change the value to either true or false depending on whether you want read-only mode or not.

  • Save the file.

  • Restart the Studio using start_sonar_finder.sh or start_sonar_finder.bat depending on your operating system.

When the Studio is running in read-only mode you must provide a Studio database when you connect (see Studio database).

Local Only Mode

JSONStudio listens for connections from any host. You can however configure it to listen to the localhost only (i.e. to only allow connections to the Studio from the host where it is installed). To configure local only mode follow these steps:

  • Shutdown the Studio using stop_sonar_finder.sh or stop_sonar_finder.bat depending on your operating system.
  • Edit conf/server.xml under your installation directory (perhaps make a copy of the file before editing it for recovery purposes).
  • Look for the XML element:: <Connector port=”8443” protocol=”HTTP/1.1” SSLEnabled=”true” ...
  • Add an attribute within this element of the form address=”127.0.0.1”. The connector element will now look like:: <Connector address=”127.0.0.1” port=”8443” protocol=”HTTP/1.1” SSLEnabled=”true” ...
  • Save the file.
  • Restart the Studio using start_sonar_finder.sh or start_sonar_finder.bat depending on your operating system.

Configuring for Embedded URLs in Dashboards and Applications

The Gateway can be used by external portals and Web applications directly - see the Using the Gateway to Build Dashboards and Applications tutorial.

In order for such embedding to be allowed you need to re-config the Gateway’s tomcat server to allow for X-FRAME-OPTIONS. By default, as a security measure, the Gateway is configured to prevent clickjack attacks and does not allow it’s pages to be referenced from another source. Therefore, if you try to embed URLs you will receive a denied error when accessing a Gateway URL of the form:

Load denied by X-Frame-Options: .... does not permit cross-origin framing.

You can enable such embedding by shutting down the Gateway and editing the jsonar/sonarFinder/WEB-INF/web.xml file in your installation directory. To allow referencing from all sites add:

<context-param>
   <param-name>x.frame.options</param-name>
   <param-value>ALLOWALL</param-value>
</context-param>

To allow referencing from certain sites replace the ALLOWALL with ALLOW_FROM <your uri>.

For more information on X-FRAME-OPTION see https://developer.mozilla.org/en-US/docs/Web/HTTP/X-Frame-Options.

The default value used if you do not specify this context-param is SAMEORIGIN.

CSRF Filter

The Studio comes with Cross-Site Request Forgery (CSRF) protection in the form of a filter that looks at every request and makes sure it has the right nonce. This is often needed by corporate security mandates but sometimes can detract from a user experience (e.g. using back button). If you wish to display this feature follow these steps:

  • Shutdown the Studio using stop_sonar_finder.sh or stop_sonar_finder.bat depending on your operating system.

  • Edit jsonar/sonarFinder/WEB-INF/web.xml under your installation directory (perhaps make a copy of the file before editing it for recovery purposes).

  • Remove the following XML elements:

    <filter>
       <filter-name>CsrfPreventionFilter</filter-name>
       <filter-class>org.apache.catalina.filters.CsrfPreventionFilter</filter-class>
       <init-param>
           <param-name>entryPoints</param-name>
           <param-value>/openChart.xhtml,/openTable.xhtml,/diff.xhtml,/runner.xhtml,/errorpage.xhtml,/login.jsf,/login.xhtml,/javax.faces.resource/theme.css.xhtml,/javax.faces.resource/theme.css.jsf,/javax.faces.resource/primefaces.css.jsf,/javax.faces.resource/jquery/jquery.js.jsf,/javax.faces.resource/primefaces.js.jsf,/javax.faces.resource/jquery/jquery-plugins.js.jsf,/sonar.css,/images/sonarWaves_base.jpg,/javax.faces.resource/images/ui-icons_333333_256x240.png.jsf,/javax.faces.resource/primefaces.css.xhtml,/javax.faces.resource/jquery/jquery.js.xhtml,/javax.faces.resource/primefaces.js.xhtml,/javax.faces.resource/jquery/jquery-plugins.js.xhtml,/javax.faces.resource/images/ui-icons_333333_256x240.png.xhtml,/javax.faces.resource/spacer/dot_clear.gif.jsf,/javax.faces.resource/spacer/dot_clear.gif.xhtml,/javax.faces.resource/images/ui-icons_ffffff_256x240.png.jsf,/javax.faces.resource/images/ui-icons_ffffff_256x240.png.xhtml,/javax.faces.resource/messages/messages.png.jsf,/javax.faces.resource/messages/messages.png.xhtml</param-value>
       </init-param>
       <init-param>
           <param-name>nonceCacheSize</param-name>
           <param-value>2000</param-value>
       </init-param>
    </filter> <filter-mapping>
    <filter-name>CsrfPreventionFilter</filter-name>
        <url-pattern>*.jsf</url-pattern>
        <url-pattern>*.xhtml</url-pattern>
    </filter-mapping>
    
  • Restart the Studio using start_sonar_finder.sh or start_sonar_finder.bat depending on your operating system.

Permissions required for Python tools

Each of the Python tools makes connections to databases. The profiler extracts documents from the profiled databases and saves them into a central repository. The differ reads documents from collections in one or more databases and saves the diffs into a database. The sampler reads collections from a datbase and saves metadata into a collection in a database.

The following table summarizes the permissions you need to grant in order for these tools to work for both the input and the output database used by each tool (can be the same database of course):

  SonarDiffer SonarSampler SonarProfiler SonarResourceManager      
  Output Inputs Metadata Inputs Output Profiled DB Managed mongod & mongos
Read X X X X X    
Write X   X   X    
Admin           X X

Table Of Contents

Previous topic

Working with SonarProfiler

Next topic

Advanced Topics

Copyright © 2013-2016 jSonar, Inc
MongoDB is a registered trademark of MongoDB Inc. Excel is a trademark of Microsoft Inc. JSON Studio is a registered trademark of jSonar Inc. All trademarks and service marks are the property of their respective owners.