Both NEOpoint and NEO provide data services so you can easily load data directly into your favourite third party tool such as Excel, R, Matlab, Python. All these tools offer a single command “web query” which takes a URL. Each tool expects a different data format, either XML, JSON or CSV so NEOpoint and NEO support all these formats. The main difference between data services from NEO and NEOpoint is as follows:
An example of two equivalent URL paths is shown below:
As with all the other tools data services allows you to load multiple data sets automatically. We have an XLSM file that demonstrates a simple case of loading 5 min price and demand from two separate reports. The macro is easy to modify to load any report, time period or instances. Excel sample
Excel expects web query data to be in XML format. You can either load it manually, or if this is something you do on a regular basis you may want to create a macro. The advantages of macros is that you can instantly load any number of live data sets and perform calculations on them all with the press of a button.
To start, with open NEOpoint and run the required report and then click the “Links” button, select and copy XML link. Sample XML link: http://www.neopoint.com.au/Service/Xml?f=104+Bids%5cRegion+Merit+Order+Stack+5min&period=Daily& from=2015-10-13&instances=Generator;NSW1§ion=0&key=as234dkj XML link has several parameters that you can customize.
To manually load a result follow these steps:
The big advantage of data services is automatically loading results. This makes it easy to create spreadsheets that with the click of a button will load several data sets and perform calculations. The macro text below shows how to load data automatically. You can simply copy the text for the macro and paste into your own copy of an Excel spreadsheet, modifying the URL as required. You can also set where you want the data to be loaded and load additional data sets in the same macro by repeating the XmlImport line with the required URL for each additional data set. To understand the macro we also need to explain two other lines of code:
The sample macro code is shown below.
Sub Macro1() ' ' Get prices ' For Each XmlMap In ActiveWorkbook.XmlMaps XmlMap.Delete Next Columns("G:H").ClearContents ActiveWorkbook.XmlImport URL:= _ "http://www.nempoint.com/Service/Xml?f=101+Prices\Dispatch+Prices+5min&period=Daily&from=2015-10-13&instances=§ion=0&key=iesys123" _ , ImportMap:=Nothing, Overwrite:=True, Destination:=Range("$G$1") End Sub
R is a very popular open source language based on the S language and used for statistical data analysis. The simplest way to load data is via the CSV format. Sample code below loads 5min price and plots it.
w=read.csv(file="http://www.nempoint.com/Service/Csv?f=101+Prices%5cRegion+Prices+5min&period=Daily&from=2015-05-03&instances=NSW1§ion=-1&key=abc123def") plot(w$DateTime, w$NSW1.Price.5min) lines(w$DateTime, w$NSW1.Price.5min) rdate<-strptime(w$DateTime, "%Y-%m-%d %H:%M:%S" ) plot(rdate, w$NSW1.Price.5min) lines(rdate, w$NSW1.Price.5min)
Python was probably the best of the tools we reviewed! Best of all its free and very widely supported with a large number of libraries like numpy, SciPy, Matplotlib, Pandas One of the best Python IDEs is PTVS from Microsoft with the Visual Studio Community Edition (all free). See all the details here: https://pytools.codeplex.com/wikipage?title=PTVS%20Installation . The screen shot below shows the PTVS user interface with some sample code for reading a NEOpoint data service.
The Python source code for loading some data is shown below:
import urllib2 import csv import dateutil import urllib2 import pyparsing import matplotlib.pyplot as plt import datetime as dt import dateutil.parser as parser url = 'http://nempoint.com/service/csv?key=abc123def§ion=0&f=101+Prices%5cDispatch+Prices+5min&period=Daily&from=2015-04-09&instances=' f = urllib2.urlopen(url) cr = csv.DictReader(f) dat =  price =  for row in cr: dat.append(parser.parse(row['DateTime'])) price.append(float(row['Price_5min'])) fig = plt.figure() plt.plot(dat, price) plt.show()
The chart resulting from running that Python code is shown below.