Please supply missing half-hourly data on generation at Huntly for September 2020
Nicky McLean made this Official Information request to Transpower New Zealand Limited
The request was successful.
From: Nicky McLean
Dear Transpower New Zealand Limited,
Firstly, the context.
Via its website https://www.emi.ea.govt.nz/Wholesale/Dat... and its sub-directories the Electricity authority provides links to monthly files containing data on the half-hourly generation and consumption of electricity at various locations, in particular for the various generators at the Huntly power station site. The data for September 2020 are missing, that is, they are not in the relevant monthly data files as distinct from being present but with all zero values. Oddly however, the data series giving the total generation at Huntly is present, and its values are not zero.
Obviously, if there is a problem with data supplied by the Electricity authority the issue should be raised with them, and this occurred to me too. Their response was that since they do not have the requested information, they do not have to supply it, and evidently, they feel no urge towards making an effort to publishing complete or consistent or correct data series and so will expend no effort to acquire the missing data. The tedious details can be found via https://fyi.org.nz/request/19587-please-... and the key part relevant here is in the reply from Mrs. Sarah Gilles (dated 28'th June 2022):
"The metered data published on EMI is as received from Transpower. "
Missing data may be found under an unexpected name (this has happened before), or might be found in other data file collections (the Electricity authority has hordes, a Library of Babel as per Jorge Luis Borges): whatever, but clearly, they deny having any responsibility and finger Transpower... Accordingly,
Second part: the request is for the half-hourly information on generation at Huntly in September 2020 by the main power station (the four individual steam turbines), the "Peak" generator (a 40MW gas turbine only) and the third installation, the 385MW gas turbine plus steam turbine (using exhaust heat from its gas turbine)
Ordinarily, the file supplied by the Electricity authority, 202009_Unit_level_generation.csv in this case would contain data series with name codes HLY2201.UN1.GENE.kWh.I.F, HLY2201.UN2.GENE.kWh.I.F, HLY2201.UN3.GENE.kWh.I,F, HLY2201.UN4.GENE.kWh.I.F, for "P40" HLY2201.E3P.GENE.KWh.I (sometimes HLY2201.UN6.GENE.kWh.I.F, sigh), and for the third HLY2201.E3P.GENE.KWh.I.F (sometimes HLY2201.UN5.GENE.kWh.I.F, sigh) and these names do appear in the files for the months before and since. On the other hand, the data for the total is provided in file 202009_Grid_import.csv with the name HLY2201.GENE.GG.GENE.kWh.I.F and so there is no call to supply those data.
It is possible that the data you have employs different code names, and is stored in a different format. Whatever is convenient for you will do, provided I can identify the correct names and get the units right - KW, KWh, MW, etc.
Yours faithfully,
Nicky McLean
From: Rochelle Furneaux
Transpower New Zealand Limited
Dear Nicky,
Further to your Information Act request, please find attached the data you
have requested.
Kind regards,
ROCHELLE FURNEAUX
Corporate Counsel IST
Transpower New Zealand Ltd
Waikoukou, 22 Boulcott Street, PO Box 1021, Wellington
P +64 4-590 6168
M +64 21 643 654
[1]transpower.co.nz
CAUTION: This message and any attachments contain information that is
CONFIDENTIAL and may be LEGALLY PRIVILEGED. If you are not the intended
recipient any use, disclosure or copying of this email message or
attachments is strictly prohibited. If you have received this message in
error, please notify the sender immediately and erase all copies of this
message and attachments.
References
Visible links
1. https://aus01.safelinks.protection.outlo...
From: Nicky McLean
Dear Ms. Furneaux,
Thank you for your reply, and I'm sorry for taking so long to notice. This was because the fyi.org system sent me no notification (or at least, none that I saw), and only this weekend did I remember to wonder what had happened, and had a look, and beheld your response. One ought not rely on computer systems to be reliable...
The data in the spreadsheet do indeed fill the hole, and match with data for the adjacent months, so, success. Some oddities cause a little difficulty. The standard in metrology is to record times of the making of a measurement, and so the times would ordinarily be of the end of the half-hour since a measurement can't be made at the start of its half-hour, only at its end. So, quoting the starting times is a bit odd.
The column names don't much matter, as this is a one-off deed, but "Rankines (Total)" is not so helpful - the parentheses cause trouble in names for computer use, and a consistent approach would have been to name that column "HLY Rankine Total" - but no real problem is presented as a human is messing about.
The units of the measurements have not been stated, but appear to be in megawatts going by the various maxima and the known capacities of the various generators. In particular, the units are not megawatt-hours in a half-hour.
Daylight saving causes more trouble than everything else put together, and here, your choice of start times is unfortunate as your data for the shrink day of Sunday 27'th September gives 2:00 for the start time of a half-hour's datum for 2:00 to 2:30. Alas, this time span does not exist as 2:00 becomes 3:00 at a stroke of the legislative pen. My data collation process attempts to recognise the lurches of daylight saving changeovers in time series, and some data suppliers do not follow the lurches while others do, and in this case, my scheme guessed that there was no time lurch in the incoming data stream and thus erred for your data. Changing 2:00 to 3:00 for that day meant that the lurch was observed to be present and all went well.
In short, some minor messing about resolved these stumbles.
It so happens that the data series supplied via the Electricity authority did not omit data for the total power generated at Huntly, and a comparison of your data's totals against theirs gave a correlation coefficient of 0·992819 which looks good, but, a plot of the difference between the two shows a surprising discrepancy of hundreds of megawatts at the start of Thursday 17'th September 2020. .Generators HLY U6 and HLY Rankine 2 are shown at zero, but starting with the first value for the 17'th, the others are constant for nineteen consecutive values, showing HLY U5: 334·280364990234, HLY Rankine 1: -4·45947027206421, and HLY Rankine 4: 191·707107543945. By contrast, the other total wobbles about in the usual manner.
Offering numbers to thirteen digits of precision seems to be a modern foible, and the chance that actually-measured data would be accurate to that extent is, well, nil. So, it looks like some of your data are corrupt, and all but the first value of a stutter sequence should be replaced by ? for "No datum" rather than mislead with an incorrect datum.
There are some other discrepancies, but they are much smaller. The data for the Electricity authority's version contain no fractional digits for example.
But most of the data are good. Thanks again.
Yours sincerely,
Nicky McLean
From: Rochelle Furneaux
Transpower New Zealand Limited
Thank you for your reply. I have passed this on to the part of the business that deals with this data.
-----Original Message-----
From: Nicky McLean <[FOI #21178 email]>
Sent: Monday, 19 December 2022 12:02 am
To: Rochelle Furneaux <[email address]>
Subject: Re: Half hourly data on generation at Huntly for September 2020
Cyber Security Warning: This sender is from outside of the organisation. Please be cautious when opening the links or attachments.
Dear Ms. Furneaux,
Thank you for your reply, and I'm sorry for taking so long to notice. This was because the fyi.org system sent me no notification (or at least, none that I saw), and only this weekend did I remember to wonder what had happened, and had a look, and beheld your response. One ought not rely on computer systems to be reliable...
The data in the spreadsheet do indeed fill the hole, and match with data for the adjacent months, so, success. Some oddities cause a little difficulty. The standard in metrology is to record times of the making of a measurement, and so the times would ordinarily be of the end of the half-hour since a measurement can't be made at the start of its half-hour, only at its end. So, quoting the starting times is a bit odd.
The column names don't much matter, as this is a one-off deed, but "Rankines (Total)" is not so helpful - the parentheses cause trouble in names for computer use, and a consistent approach would have been to name that column "HLY Rankine Total" - but no real problem is presented as a human is messing about.
The units of the measurements have not been stated, but appear to be in megawatts going by the various maxima and the known capacities of the various generators. In particular, the units are not megawatt-hours in a half-hour.
Daylight saving causes more trouble than everything else put together, and here, your choice of start times is unfortunate as your data for the shrink day of Sunday 27'th September gives 2:00 for the start time of a half-hour's datum for 2:00 to 2:30. Alas, this time span does not exist as 2:00 becomes 3:00 at a stroke of the legislative pen. My data collation process attempts to recognise the lurches of daylight saving changeovers in time series, and some data suppliers do not follow the lurches while others do, and in this case, my scheme guessed that there was no time lurch in the incoming data stream and thus erred for your data. Changing 2:00 to 3:00 for that day meant that the lurch was observed to be present and all went well.
In short, some minor messing about resolved these stumbles.
It so happens that the data series supplied via the Electricity authority did not omit data for the total power generated at Huntly, and a comparison of your data's totals against theirs gave a correlation coefficient of 0*992819 which looks good, but, a plot of the difference between the two shows a surprising discrepancy of hundreds of megawatts at the start of Thursday 17'th September 2020. .Generators HLY U6 and HLY Rankine 2 are shown at zero, but starting with the first value for the 17'th, the others are constant for nineteen consecutive values, showing HLY U5: 334*280364990234, HLY Rankine 1: -4*45947027206421, and HLY Rankine 4: 191*707107543945. By contrast, the other total wobbles about in the usual manner.
Offering numbers to thirteen digits of precision seems to be a modern foible, and the chance that actually-measured data would be accurate to that extent is, well, nil. So, it looks like some of your data are corrupt, and all but the first value of a stutter sequence should be replaced by ? for "No datum" rather than mislead with an incorrect datum.
There are some other discrepancies, but they are much smaller. The data for the Electricity authority's version contain no fractional digits for example.
But most of the data are good. Thanks again.
Yours sincerely,
Nicky McLean
-----Original Message-----
Dear Nicky,
Further to your Information Act request, please find attached the data you have requested.
Kind regards,
ROCHELLE FURNEAUX
Corporate Counsel IST
Transpower New Zealand Ltd
Waikoukou, 22 Boulcott Street, PO Box 1021, Wellington
P +64 4-590 6168
M +64 21 643 654
[1]transpower.co.nz
CAUTION: This message and any attachments contain information that is CONFIDENTIAL and may be LEGALLY PRIVILEGED. If you are not the intended recipient any use, disclosure or copying of this email message or attachments is strictly prohibited. If you have received this message in error, please notify the sender immediately and erase all copies of this message and attachments.
References
Visible links
1. https://aus01.safelinks.protection.outlo...
-------------------------------------------------------------------
Please use this email address for all replies to this request:
[FOI #21178 email]
Disclaimer: This message and any reply that you make will be published on the internet. Our privacy and copyright policies:
https://aus01.safelinks.protection.outlo...
If you find this service useful as an Official Information officer, please ask your web manager to link to us from your organisation's OIA or LGOIMA page.
-------------------------------------------------------------------
hide quoted sections
Things to do with this request
- Add an annotation (to help the requester or others)
- Download a zip file of all correspondence