Exporting historical data from ThinkorSwim for external analysis

[korygill]

Hi korygill, is there a version of the shell script that would work on Mac? I am able to download the CSV file but I am not a Windows user and don't understand the shell script. appreciate any help you can provide. Thanks!

I don't have a mac, but you can install powerhsell on it like: docs.microsoft.com/en-us/powershell/scripting/install/installing-powershell-core-on-macos?view=powershell-7 which may or may not work for you. Maybe there are others on this forum that use a mac and can assist.

 

[TradeCloserApp]

@korygill

If I'm on a daily chart & I set my start to 1700 startTime & endTime 930 open (show below), it yields the same result as 820 & 1600 as my times.

declare upper;
declare once_per_bar;

#input startTime = 820; #hint startTime: start time in EST 24-hour time
#input endTime = 1600; #hint endTime: end time in EST 24-hour time
input startTime = 1700; #hint startTime: start time in EST 24-hour time 
input endTime = 930; #hint endTime: end time in EST 24-hour time

def adjStartTime = startTime;# - 1;
def adjEndTime = endTime;# - 1;

def agg = GetAggregationPeriod();

# we use a 1 bar offset to get orders to line up, so adjust for that here
def marketOpen = if agg >= AggregationPeriod.DAY then 1 else if SecondsTillTime(adjEndTime) >= 60 and SecondsFromTime(adjStartTime) >= -60 then 1 else 0;

 

[korygill]

A daily chart can only gather data about each Day. You need to use intraday charts to get a subset. If you cant see the exact bars you want on a chart, we can't report on that bar's data. Hope that makes sense...

 

[TradeCloserApp]

A daily chart can only gather data about each Day. You need to use intraday charts to get a subset. If you cant see the exact bars you want on a chart, we can't report on that bar's data. Hope that makes sense...

Makes complete sense, but was also my fear. (was hoping for a hack)
Thanks again for all you! Great learning experience.

 

[Picard]

I'd like to get intraday 1 minute data for a date range of June 26, 2016 to July 2, 2016. How can I do this with your scripting technique. There may be times that I will need historical date range intraday data for other projects, so I will find this very useful. Your help is appreciated.

 

[korygill]

I'd like to get intraday 1 minute data for a date range of June 26, 2016 to July 2, 2016. How can I do this with your scripting technique. There may be times that I will need historical date range intraday data for other projects, so I will find this very useful. Your help is appreciated.

You can do that by using the OnDemand feature in TOS and replay a historical chart. Other alternative is a service like Kibot or other quote providers.

 

[Picard]

Problem With TOS Data Export

I changed the PS Execution Policy using this video.

How To Change The PowerShell Script Execution Policy

Here is my video recording of the problem with the procedure. Am I using the right indicator script for the data conversion?
https://www.screencast.com/t/Ak6VxslYCSm
I placed my Get-SOHLCP.ps1 and the exported TOS stradegy report in the Z:\tos-data\ folder

I tried the following commands in PowerShell

$data = .\Get-SOHLCP.ps1 .\StrategyReports_SPYV_2016.csv

$data = Z:\tos-data\Get-SOHLCP.ps1 Z:\tos-data\StrategyReports_SPYV_2016.csv

No converted CSV file was created. I don't know what I missed that this routine didn't work. I may not have the same Get-SOHLCP.ps1 script that you show in your video. I'm having trouble copying your scripts from my computer screen. Would it be possible for you to post them as text files in the forum? Please help.

 

[korygill]

@Picard , are you sure after running the script a new file was not generated in your z:\tos-data directory? From what I can tell, you have the right script and ran it properly.

 

[Picard]

In the previous Screencast video I showed you the kg_EveryTickSOHLCPSTRATEGY.ts that I imported into TOS using your link https://tos.mx/1FmyWu4, but I should have included showing you the script for Get-SOHLCP.ps1 that I'm using so that you can confirm what I'm using is correct. Please review my "Review Of Get-SOHLCP.ps1 Script" and let me know if this is the same script that you are using in you video. I haven't found a way to upload a file into this forum. So far I have not been able to get my stradegy report to convert. Converted files do now show up in my Z:\tos-data folder. I'm following your instructions the best that I can, but if there is some other Windows 10 setting that I haven't done, please let me know. Here is the link to my Screencast video.

https://www.screencast.com/t/fP6uGTs3PBq1
Here are the files that I am using.
https://anonfiles.com/B4yaR0r8o6/TOS_Data_Export_Scripts_rar

 

[korygill]

that site you gave wants me to install a bunch of stuff to get that file. not gonna do it. a onedrive or gdrive or dropbox would be ok.

this post (#31) has all the latest working scripts:
https://usethinkscript.com/threads/...m-for-external-analysis.507/page-2#post-17387

 

[Picard]

I have difficulty copying text from my computer screen, so I'm not sure that I copied the code correctly. I posted the file on another storage website, but if its not too much trouble please post your code on a storage website. This one is very good, free and has high capacity. https://mega.nz
Here is my file https://mega.nz/file/xhlnhSAY#uUU6m5jUVLvrjFnJN55k1z__hSe_sar6SrwXwLn9k5I

I cannot think of any other reason that I cannot get this to work, except maybe I copied the code incorrectly. Sorry for the fuss and I appreciate your help. Did you view the Screencast video?

 

[korygill]

Your code in that rar file worked fine.

See also this post #20 for how to use it.
https://usethinkscript.com/threads/...nkorswim-for-external-analysis.507/post-14606
Run from directory the script and csv file are both in....

See the output:
.\Get-SOHLCP.ps1 -File .\StrategyReports_SPYV_2016.csv

Put output in a variable:
$data = .\Get-SOHLCP.ps1 -File .\StrategyReports_SPYV_2016.csv

Save to csv:
$data | ConvertTo-Csv -NoTypeInformation | Out-File -FilePath .\SPYV-Data.csv

 

[Picard]

Thanks korygill,

That worked perfectly by executing each line of commands in sequence. Is possible to put those commands into a batch file and execute it in PowerShell? I've been having trouble importing the resulting data into my trading platforms and I think it might be that the values for the OHLCP are strings and that the output would work if they are numbers.

Your code works great. If you could help me convert those OHLCP strings into numbers, that the output file would be easier to import into other programs. Thank you very much for your help.

The exported data needs to be in this format.

 

[designatedrhythm]

What would be the script to Plot Vwap and show the value for each candle. That way I could export each candle vwap value to excel.

 

[OptionsPirate]

Does anyone know if it is possible to get any time frame historical data from TOS to download into a spread sheet?

Thanks...

 

[Derek]

Thank you very much for your contributions, @korygill. I'm going to try to do it.

 

[JonR]

If anyone is interested, I adjusted the thinkScript strategy and the Powershell script to include volume. I have a screenshot but don't know how to include that in posts to the forum.

 

[Picard]

JonR,

I also adapted the script to include volume; however, in order for most trading platforms to import this data the numerical values have to be in a numerical format not in a text format as this script generates. You can tell the format by seeing that the numbers are enclosed in quotation marks. Also the date/time field needs to be separated into a date field and a time field separately, not date & time combined. Possibly my previous post didn't clarify that. Have you tried to import your data into a trading platform yet?

Script for exporting volume.
https://mega.nz/file/4tckTQSB#voXExrndNk_PvDMC8880XjioVl946FVAu_i_uRZ4HZ0

 

[Picard]

Does anyone know if it is possible to get any time frame historical data from TOS to download into a spread sheet?

Thanks...

Jmo63,

I show how to do that this video. As for getting into a spreadsheet, the spreadsheet program should allow for importing the CSV file; however, the fields that have numbers are in text format, not numerical.

https://www.screencast.com/t/beLka3XBIdJr

 

[leecharles]

Hi there!

Is there any way to export the historical implied volatility data from the ImpVolatility study to a CSV file?

I'm trying to calculate the expected movement of a stock for earnings using the following formula: Stock Price x IV x sqrt(# of days / 365). I realize I can get the expected movement of a stock by looking at the ATM straddle. However, I also need to get historical expected moves and actual moves. Doing so means I need historical options data, but it's quite expensive to purchase from other sources. All I really need is just the implied volatility value anyway.

Any creative solutions to export this data would be much appreciated!

Below is the ImpVolatility study highlighted in blue. All of these values is what I'm trying to export.