QueryTool

Advanced web based querying service of energy data sets

QueryTool is a powerful database interrogation tool that connects to our cloud based data services.
It gives you access to the full history of NEM, STTM, VicGas, and BOM data.

QueryTool main features

  • Its a desktop app that connects over the web to IES data servers hosted in Google Cloud Services in Australia.
  • Very fast and handles huge data queries over millions of rows.
  • Data service lets you automate the running of queries and collecting data in other programs like Excel, Matlab, R and Python.
  • Unlimited use. Install it at work or home and have ready access.
  • Access the full history of Australian NEM, STTM, DWGM, and BOM data.
  • Instantly sort millions of rows on any column. Sorting is handled locally so its very fast.
  • Export as CSV.
  • Runs standard MySQL 5.7 queries.

SQL Data Service

  • You don't need to run QueryTool to access the cloud based QueryTool service. You can run SQL queries from most third party applications like Python, Excel, PowerBI, Matlab and R.
  • Data is returned as CSV which can be loaded by most third party programs.
  • The service can handle large queries > 10M rows.
  • The sample code on the right shows how to run a query and import the data into Python.
  • See Data Services for more details.
# Imports
import requests
import io
import pandas

# Query to run
query = '''select `SETTLEMENTDATE`, `RUNNO`, `REGIONID`, `PERIODID`, `RRP`, `EEP`,
`INVALIDFLAG`, `LASTCHANGED`, `ROP`, `RAISE6SECRRP`, `RAISE6SECROP`,
`RAISE60SECRRP`, `RAISE60SECROP`, `RAISE5MINRRP`, `RAISE5MINROP`,
`RAISEREGRRP`, `RAISEREGROP`, `LOWER6SECRRP`, `LOWER6SECROP`, `LOWER60SECRRP`,
`LOWER60SECROP`, `LOWER5MINRRP`, `LOWER5MINROP`, `LOWERREGRRP`, `LOWERREGROP`,
`PRICE_STATUS` from mms.tradingprice where SETTLEMENTDATE between '2018-11-01' and '2018-11-02' limit 0, 10000'''

# payload for POST request
url = 'http://www.nempoint.com/data/query'
data = { 'query': query, 'key':'abc123', 'format':'csv'}

# Send HTTP request
response = requests.post(url, data=data)# Note do NOT use params=data as this serialises to the URL!

# Parse the response
data = pandas.read_csv(io.StringIO(response.content.decode('utf-8')))

Other features

  • Includes a schema view that shows you meta data COMMENTS.
  • Events view shows all queries run so far and allows copying them.
  • Can open multiple tabs so you can run multiple queries and quickly switch between result sets.
  • Column widths are automatically sized.
  • Columns are colourised based on data type.

Large Data Sets

Specially designed to retrieve very large data-sets, handling millions of rows of data with ease.

It makes use of advanced caching and pre-fetching techniques, allowing you to scroll through enormous data results in real time.

All the data you need

QueryTool service open a door to numerous Energy Market database like NEM, VicGas (DGWM), STTM, WEM WA, as well as other related data like BOM. All of these data are stored in Google Cloud Platform for high reliability to ensure maximum uptime.

Data services

Use the data services feature to also run queries directly from your favourite analysis tools like Excel, Power BI, Matlab, R and Python.

Query and Schema Views

At the click of a button, Query Tool enables you either examine the metadata for any table, or view a query and its results on the same page.

Export Data

After running a query, you have the option to export your data to a CSV file.

Sorting/Searching Data

The results of any query can be sorted by any column without re-executing the query, and you can easily search using any desired string.