Last time I left off at ditching Tradingview as a quote source. These things happen, and I learned quite a lot along the way, so I’m not troubled by it. Turning locally, as in my own hard drive, I began sifting through the data that gets archived by my chart provider.
Fortunately they aren’t super-protective of their quotes, so the files were essentially text data with some custom file extensions on the end. I began Python-ing a way to get these things filtered and saved in the proper folder for my Teardown mod.
The data is quite orderly:
Date, Time, Open, High, Low, Last, Volume, # of Trades, OHLC Avg, HLC Avg, HL Avg, Bid Volume, Ask Volume 2022/4/4, 13:22:45.0, 4561.50, 4561.50, 4561.50, 4561.50, 3, 3, 4561.50, 4561.50, 4561.50, 0, 3
After a bit of filtering/detection, I had it doing things like this:
Last line in file: ESM22-CME-BarData.txt is: 2022/4/4, 20:32:40.0, 4571.25, 4571.25, 4571.25, 4571.25, 3, 3, 4571.25, 4571.25, 4571.25, 3, 0 Symbol: ES Last price: 4571.25 Volume: 3 File size in bytes: 12587
(I’m counting filesize so I can wipe it when it hits a certain size, to keep search times down for some functions.)
One main task I had to figure out was filtering based on futures symbol. Most symbols follow the convention of:
<symbol><monthcode><YY>
As with anything, there are a few exceptions where the year/month is flipped, but in aggregate my initial function seemed to do the trick.
I realized later on that this first pass wasn’t good enough – it wasn’t behaving EXACTLY as I needed, so it made sense to go through and refactor it into a function (or series of them), that worked how I wanted. Sometimes being “cute” and trying to do too many things at once creates technical debt that bites you in the ass when you least expect it.
After some pseudo-code restructuring to actually plan the flow of what I was trying to do, I came up with this result:
I set up a test where I threw it every symbol I had in my local directory to make sure it could handle things:
So far, so good! It was nice being able to chuck anything at it, even the dollar-sign prefix ones, and have it handle things in a sane way.
I now had to plan out the pseudo-code for making the main polling loop work. I’m not a Python expert by any means, so I just worked from some examples and adapted it to my purpose. After thinking a bit, I managed to get this flow:
This happens to me sometimes, I’m humming along and Python-ing my way to a given goal, and something gets thrown in my way that I didn’t quite expect.
Debugging isn’t glamorous, there aren’t any hacker-esque (in the hollywood sense) terminals with interesting things being scrolled/displayed/animated on them. Its just my boring Notepad++ open to a file and the shell where I’m running Python commands.
At one point, things were silently failing.
It could be my fault – I’m not an expert, as I said – so I could be structuring things too deeply in terms of logic IF .. THEN or just calling a bunch of functions like a fool. The result was I had to place some Debug statements like — print(“Did I even get to this point?”) in the code to figure out WHERE it was dying.
Not fun, in the least.
At this point, I’m nearly at the end of the Python part of implementing this mod — the rest of the steps are to get the voxel model encoded and saved to the proper directory, then in Teardown have it get read/decoded by a lua script.
So very close…. more to come.