sum vs fold

[markt]

I'm a newbie to thinkscript. As such I decided to write my own little piece of code to calculate the std of a set of data. To do this I used the sum function. However, when I compared the results to the inbuilt std formula there was a difference between my calculation and that of the inbuilt function. When I looked at the code for the inbuilt function (provided in the thinkscript help pages) I noticed that it was using the fold command rather than the sum command.

I must admit I am totally lost as to why my code doesn't give the same answer. I have posted it below. Have I just made a silly booboo or is there something fundamentally wrong with my understanding (or both?). (I was expecting the output of my code to be a straight line with yy=0.)


script getstd_diff {
input y = close;
input length = 50;

def av_y= Average(y,length);
def mystd = Sqrt( Sum(Sqr(y-av_y),length)/length );
def inbuilt = StDev(close,length);
plot difff=mystd-inbuilt;
}


declare lower;
input length = 50;
plot yy=getstd_diff(close,length);

 

[halcyonguy]

how are others supposed to compare your code, when you don't provide the fold code ( or link) to it, to compare to ?

 

[markt]

my apologies... here is the stdDev help function code copied from the thinkscript reference manual. As I understand it, StDev and StDevTS give identical results. Therefore I assume that the inbuilt calculation of standard deviation is equivalent to the fold based expression "Sqrt( (fold i = 0 to length with SD do SD + Sqr(GetValue(data, i) - avgData) ) / length);"


script StDevTS {

input data = close;

input length = 12;

def avgData = Average(data, length);

plot StDevTS1 = Sqrt( (fold i = 0 to length with SD do SD + Sqr(GetValue(data, i) - avgData) ) / length);

}



declare lower;

input length = 10;

plot StDev1 = StDev(close, length);

plot StDev2 = StDevTS(close, length).StDevTS1;

 

[halcyonguy]

no need for apologies. just want people to think about providing as much info as
possible , to make it easier for others to help them.
i looked at this site, printed it out and compared to your code. i can look at it in tos in a couple hours.
https://tlc.thinkorswim.com/center/reference/thinkScript/Functions/Statistical/StDev

your code has a sum in it and doesn't match the example code? maybe that is the problem.

plot StDevTS2 = Sqrt(Average(Sqr(data), length) - Sqr(Average(data, length)));

i think you may have to pull the built in function out of your script and then do the comparison.

 

[SleepyZ]

I'm a newbie to thinkscript. As such I decided to write my own little piece of code to calculate the std of a set of data. To do this I used the sum function. However, when I compared the results to the inbuilt std formula there was a difference between my calculation and that of the inbuilt function. When I looked at the code for the inbuilt function (provided in the thinkscript help pages) I noticed that it was using the fold command rather than the sum command.

I must admit I am totally lost as to why my code doesn't give the same answer. I have posted it below. Have I just made a silly booboo or is there something fundamentally wrong with my understanding (or both?). (I was expecting the output of my code to be a straight line with yy=0.)


script getstd_diff {
input y = close;
input length = 50;

def av_y= Average(y,length);
def mystd = Sqrt( Sum(Sqr(y-av_y),length)/length );
def inbuilt = StDev(close,length);
plot difff=mystd-inbuilt;
}


declare lower;
input length = 50;
plot yy=getstd_diff(close,length);

See if this helps. It matches the TOS one's you provided, however, I had to make av_y = average(close,1) instead of how you defined it. As I am not familiar with the math, perhaps you can figure it out from the following

script getstd_diff {
    input y      = close;
    input length = 10;
    def av_y     = average(close, 1);
    def mystd    = Sqrt(Sum(Sqr(Sum(close - av_y, length)), length) / length );
    def inbuilt  = StDev(close, length);
    plot difff   = inbuilt - mystd;
}

declare lower;
input length = 10;
plot yy = getstd_diff(close, length);

 

[markt]

I appreciate people getting back to me. I think I now understand what is going on. When calculating a standard deviation, we find the summation of Sqr(y_i - y_av) with i in the range (1,length) (formula given here ). It is only necessary to calculate the average value of y, y_av, once .

In the original code that I posted, I assumed that on a given bar, once y_av was calculated its value was then fixed for that bar (def av_y= Average(y,length);). However it appears that it is calculated at least 'length-1' more times if it appears within the sum function. During the summation, each calculation uses the bar of interest (with closing value = y_i) and then finds the average value over length bars up to that bar. I guess we could say that the y_av turns into a moving average when included in the sum function (i.e its value is now not fixed)

I figured this out by writing a little piece of matlab code that produces both the inbuilt standard deviation, std1, value and the mystd, std2, value produced in my original post. I have copied this below

% sign indicates a comment

x=[238.47 ,236.61, 240.00, 239.6, 238.63, 235.15, 233.82, 235.95]; % Visa close prices

%  7/9        7/8        7/7      7/6     7/2        7/1      6/30      6/29    dates


llength=4;


av1=mean(x(1:llength));  %finds average over 'llength' values


sqrsum1=0; %initialize summation

sqrsum2=0; %initialize summation


for i=1:llength

    av2=mean( x(i : i+(llength-1) )); %finds average over llength values commencing at current x value i.e moving average

    sqrsum1=sqrsum1+(x(i)- av1)^2;

    sqrsum2=sqrsum2+(x(i)- av2)^2;

end


std1 = sqrt((sqrsum1/llength))     % corresponds to inbuilt in my original post

std2 = sqrt((sqrsum2/llength))     % corresponds to mystd in my original post

At least I know now where the error was in my assumption that

def av_y= Average(y,length);
def mystd = Sqrt( Sum(Sqr(y-av_y),length)/length );

should produce the correct standard deviation. Any thoughts or comments are much appreciated

 

[pupu]

Although the result is very similar, what is the "volatility" difference, or mathematical/trading significance between code (using StdDev) compared to classic (using LinearDev)? Why use each over the other?

StdDev: Sqrt of LinearDev

plot StDevTS1 = Sqrt(
                          (
                            fold i = 0 to length
                            with SD
                            do SD + Sqr(GetValue(data, i) - avgData)
                          ) / length
                        );

LinearDev:

plot LinDevTS =
              (
                fold i = 0 to length
                with LD
                do LD + AbsValue( avgData - GetValue(data, i) )
              ) / length;