use Calculate and CalculateValues in the same study

[Spin]

Hello world

For my newest study I need to compare the values of the HMA against other MA's. So I downloaded the HMA code that the friendly MotiveWave-people were so kind to share.
This HMA uses CalculateValues and OnBarUpdate. I then added a 'Calculate' section where I calculate & draw 4 other MA's:

@Override  
  protected void calculate(int index, DataContext ctx)
          {
      
      debug("Inside calculate " );
          DataSeries series = ctx.getDataSeries();

          int EMA21HiPeriod=getSettings().getInteger(EMA_21_Hi_Period);
          //debug ("testing: Periods =  " + EMA21HiPeriod + " index " + index);
          int EMA21LoPeriod=getSettings().getInteger(EMA_21_Lo_Period);
          int EMA144Period=getSettings().getInteger(EMA_144_Period);
          int EMA544Period=getSettings().getInteger(EMA_544_Period);
                    
          // EMA's
          Double Hi21=series.ma(getSettings().getMAMethod(EMA_21_Hi_Method), index, EMA21HiPeriod, getSettings().getInput(EMA_21_Hi_Input));
          Double Lo21=series.ma(getSettings().getMAMethod(EMA_21_Lo_Method), index, EMA21LoPeriod, getSettings().getInput(EMA_21_Lo_Input));
          Double E144=series.ma(getSettings().getMAMethod(EMA_144_Method), index, EMA144Period, getSettings().getInput(EMA_144_Input));
          Double E544=series.ma(getSettings().getMAMethod(EMA_544_Method), index, EMA544Period, getSettings().getInput(EMA_544_Input));
          //Double HMA=series.ma(getSettings().getMAMethod(HMA_METHOD), index, HMAPeriod, getSettings().getInput(HMA_INPUT));
        
          // 'safety check' to see if values exist already
          if (Hi21 == null) return;
          if (Lo21 == null) return;
          if (E144 == null) return;
          if (E544 == null) return;
        
          
      debug ("Hi21 == " + Hi21);
          
 
      //set values for current bar
          series.setDouble(index, Values.EMA_21_Hi, Hi21);
          series.setDouble(index, Values.EMA_21_Lo, Lo21);
          series.setDouble(index, Values.EMA_144, E144);
          series.setDouble(index, Values.EMA_544, E544);
         
series.setComplete(index);      
    }

I cannot seem to get the Calculate and the CalculateValues functions to run at the same time in my study: either I comment the entire CalculateValues section, so Calculate works and my 4 MA's get drawn or I uncomment CalculateValues and only the HMA gets drawn.

In the study log I only see the debug-statement "Hi21=" printed once. So it seems the Calculate function is only run once (or for one bar).


What do I need to change / add to have both the 4 MA's and the HMA drawn ??

All help welcome !

 

[lndshrk]

Try adding 'super.calculateValues(ctx)' to your main CalculateValues() method and that should run the Calculate() method

// Invoke the parent method to run the "calculate" method
super.calculateValues(ctx);

 

[Spin]

Thanks for your help, lndshrk

Tried that, no change.

 

[lndshrk]

Just to double check, in the SDK example documentation they have a composite sample that might be a good reference to check your code against.

Otherwise you may want to avoid using the calculate method and write a separate method that you invoke from within CalculateValues(). I personally was getting weird results when I started using calculate for other calculations, so now I simply do not use it and call other functions as I need them.

 

[Spin]

Update: I now enalrged my 'Calculate' function to also contain the HMA-calculations. This is the end result:

 @Override

  protected void calculate(int index, DataContext ctx)
          {
    
      debug("Inside calculate " + "index = " + index);
          DataSeries series = ctx.getDataSeries();
        
          //if (!series.isBarComplete(index)) return;
    
          //Coordinate d5=new Coordinate(series.getStartTime(index), 9);
        Coordinate d0=new Coordinate(series.getStartTime(index), 20);
        Coordinate d4=new Coordinate(series.getStartTime(index), 26);
        Coordinate d=new Coordinate(series.getStartTime(index), 36);
        Coordinate d2=new Coordinate(series.getStartTime(index), 56);
        Coordinate d3=new Coordinate(series.getStartTime(index), 76);
      
        int EMA21HiPeriod=getSettings().getInteger(EMA_21_Hi_Period);
          //debug ("testing: EMA 21 Period =  " + EMA21HiPeriod + " index " + index);
          int EMA21LoPeriod=getSettings().getInteger(EMA_21_Lo_Period);
          int EMA144Period=getSettings().getInteger(EMA_144_Period);
          int EMA544Period=getSettings().getInteger(EMA_544_Period);
          //int HMAPeriod=getSettings().getInteger(HMA_PERIOD);
        
          // EMA's
          Double Hi21=series.ma(getSettings().getMAMethod(EMA_21_Hi_Method), index, EMA21HiPeriod, getSettings().getInput(EMA_21_Hi_Input));
          Double Lo21=series.ma(getSettings().getMAMethod(EMA_21_Lo_Method), index, EMA21LoPeriod, getSettings().getInput(EMA_21_Lo_Input));
          Double E144=series.ma(getSettings().getMAMethod(EMA_144_Method), index, EMA144Period, getSettings().getInput(EMA_144_Input));
          Double E544=series.ma(getSettings().getMAMethod(EMA_544_Method), index, EMA544Period, getSettings().getInput(EMA_544_Input));
          //Double HMA=series.ma(getSettings().getMAMethod(HMA_METHOD), index, HMAPeriod, getSettings().getInput(HMA_INPUT));
      
          // 'safety check' to see if values exist already
          if (Hi21 == null) return;
          if (Lo21 == null) return;
          if (E144 == null) return;
          if (E544 == null) return;
          //if (HMA == null) return;
        
      //debug ("Hi21 == " + Hi21);
        

      //set values for current bar
          series.setDouble(index, Values.EMA_21_Hi, Hi21);
          series.setDouble(index, Values.EMA_21_Lo, Lo21);
          series.setDouble(index, Values.EMA_144, E144);
          series.setDouble(index, Values.EMA_544, E544);
          //series.setDouble(index, Values.HMA, HMAvalue);
        

        
        
        
          // Drawing markers to checkplot for different entry conditions
          if (Hi21 > E144)
         { debug("Inside drawing condition " + "index = " + index);
         // debug ("drawing == ");
         MarkerInfo marker = getSettings().getMarker(MARKER9);
         if (marker.isEnabled())
         addFigure(CHECK_PLOT, new Marker(d0, Enums.Position.CENTER, marker));     
         }
        
        
    ////////////////////////////////////////////////////////////////////////////////////////////////////////////////////   
     // For HMA
         // DataSeries series = ctx.getDataSeries();
          int HMAperiod = getSettings().getInteger(HMA_PERIOD);
          int sqp = (int)Math.sqrt(HMAperiod);
          Enums.MAMethod HMAmethod = getSettings().getMAMethod(HMA_METHOD);
          int HMAshift = getSettings().getInteger(HMA_SHIFT);
          Object input = getSettings().getInput(HMA_INPUT);
        
          for(int i = HMAperiod; i < series.size(); i++)
          { debug("Inside HMA 1 " + "index = " + index);
              if (series.isComplete(i)) continue;
              Double ma1 = series.ma(HMAmethod, i, HMAperiod/2, input);
              if (ma1 == null) continue;
              ma1 = ma1*2;
              Double ma2 = series.ma(HMAmethod, i, HMAperiod, input);
              if (ma2 == null) continue;
              series.setDouble(i, Values.HMA_Tmp, ma1 - ma2);
            }
        
          int end = series.size()-1;
          int start = 0;
            //if (shift < 0) end -= shift; // calculate future values
        
          if (start <= HMAperiod+sqp) start = HMAperiod + sqp;
          int lastComplete = series.size()-1;
          if (HMAshift < 0) lastComplete += HMAshift;
        
          for(int i = start; i <= end; i++)
          { debug("Inside HMA 2 " + "index = " + index);
              if (series.isComplete(i+HMAshift)) continue;
              Double value = series.ma(HMAmethod, i, sqp, Values.HMA_Tmp);
              if (value == null) continue;
              series.setDouble(i+HMAshift, Values.HMA, value);
              series.setComplete(i+HMAshift, i >= 0 && i < lastComplete); // latest bar is not complete
            }
        
          clearFigures();
          for(int i = 1; i < series.size(); i++)
          { debug("Inside HMA 3 " + "index = " + index);
              Double ma = series.getDouble(i, Values.HMA);
              if (ma == null) continue;
//              if (doBarColor) {
//                Double prevMa = series.getDouble(i-1, Values.MA);
//                if (prevMa == null) continue;
//                if (ma < prevMa) series.setPriceBarColor(i, getSettings().getColor(Inputs.DOWN_COLOR));
//                else if (ma == prevMa) series.setPriceBarColor(i, getSettings().getColor(Inputs.NEUTRAL_COLOR));
//                else series.setPriceBarColor(i, getSettings().getColor(Inputs.UP_COLOR));
//              }
        
        
          series.setComplete(index);   
          }
}

In the study log, I now get literally hundreds of "Inside HMA 1 index = 1488", followed by "Inside HMA 2 index = 1488" and again followed by "Inside HMA 3 index = 1488".
The only MA that gets drawn is the HMA

So my code keeps running around in circles inside those "for" -statements. But why ? How can I 'break' these loops and make it calculate the other MA's ??

On a sidenote: thanks, lndshrk, for the good advice. But how would I tackle this ? How does one write a 'separate' method for the calculation of those 3 other MA's and how to invoke it from within CalculateValues ?
(my java is definitely not as good as my Python. I hope the good people at MW soon allow for plugins in other languages )

 

[lndshrk]

So - for me, the "calculate()" method has caused a ton of issues. On simple indicators with essentially one calculation, its totally fine. But I've found that with any complexity, it starts to give some very strange results. I've got zero facts on how MotiveWave handles process flow, but it appears the calculate() method does not consistently sweep through the index's like you are probably expecting. I've seen this in the debugger but I cannot tell you why. So I've migrated to calculateValues() and 'forced' the sequence through a loop like this...

      @Override
      protected void calculateValues(DataContext ctx)
      {
          DataSeries series = ctx.getDataSeries();
          int latest = series.size()-1;
          int index = latest;
          boolean updates = getSettings().isBarUpdates();
          if (index < getSettings().getInteger(SMA_Period)*2) return;
         
          //if (!series.isBarComplete(index)) return;
          // Moving Average Calculation
         
          for(int i = getSettings().getInteger(SMA_Period); i <= latest; i++) {
              if (series.isComplete(i, Values.MA)) continue;
              if (!updates && !series.isBarComplete(i)) continue;
              Util.calcSeriesMA(ctx, getSettings().getMAMethod(SMA_Method), getSettings().getInput(SMA_Input), getSettings().getInteger(SMA_Period),
                    0, Values.MA, false, getSettings().isBarUpdates());
              series.setComplete(i, Values.MA, i >= 0 && i < latest); // bars are set to complete
             
          }
         
          }

To create a method to have your calculations outside the main calculate values, you would do something like what I did here that would create multiple SMI lines.. This would go in calculateValues().

  int hlPeriod1 = getSettings().getInteger(SMI_HL1);
            int maPeriod1 = getSettings().getInteger(SMI_MA1);
            int hlPeriod2 = getSettings().getInteger(SMI_HL2);
            int maPeriod2 = getSettings().getInteger(SMI_MA2);
         
          for(int i = maPeriod2; i <= latest; i++) {
              if (series.isComplete(i, SMIVal.SMI2)) continue;
              if (!updates && !series.isBarComplete(i)) continue;
             
              smiLine(i, ctx, hlPeriod1, maPeriod1, SMIVal.SMI1, SMIValD.D1, SMIValHL.HL1, SMIValD_MA.D_MA1, SMIValHL_MA.HL_MA1); // <- SMI Calculation
              smiLine(i, ctx, hlPeriod2, maPeriod2, SMIVal.SMI2, SMIValD.D2, SMIValHL.HL2, SMIValD_MA.D_MA2, SMIValHL_MA.HL_MA2); // <- SMI Calculation
              series.setComplete(i, SMIVal.SMI2, i >= 0 && i < latest); // bars are set to complete
          }

and then outside of calculateValues() I had this..

      // Calculates the SMI line
      public void smiLine (int latest, DataContext ctx, int hlPeriod, int maPeriod, SMIVal smiFinal,
              SMIValD d, SMIValHL hl, SMIValD_MA dMa, SMIValHL_MA hlMa)
      {
       
           if (latest < hlPeriod) return;

            DataSeries series = ctx.getDataSeries();
            double HH = series.highest(latest, hlPeriod, Enums.BarInput.HIGH);
            double LL = series.lowest(latest, hlPeriod, Enums.BarInput.LOW);
            double M = (HH + LL)/2.0;
            double D = series.getClose(latest) - M;
           
            series.setDouble(latest, d, D);
            series.setDouble(latest, hl, HH - LL);
           
            if (latest < hlPeriod + maPeriod) return;
           
            Enums.MAMethod method = getSettings().getMAMethod(SMI_METHOD);
            series.setDouble(latest, dMa, series.ma(method, latest, maPeriod, d));
            series.setDouble(latest, hlMa, series.ma(method, latest, maPeriod, hl));
           
            int smoothPeriod= getSettings().getInteger(SMI_SMOOTH);
            if (latest < hlPeriod + maPeriod + smoothPeriod) return;
           
            Double D_SMOOTH = series.ma(method, latest, smoothPeriod, dMa);
            Double HL_SMOOTH = series.ma(method, latest, smoothPeriod, hlMa);
           
            if (D_SMOOTH == null || HL_SMOOTH == null) return;
            double HL2 = HL_SMOOTH/2;
            double SMI = 0;
            if (HL2 != 0) SMI = 100 * (D_SMOOTH/HL2);
            series.setDouble(latest, smiFinal, SMI);
      }

 

[lndshrk]

@Spin Also set your isComplete and setComplete statement in the loops to address the moving average series you are calculating.. otherwise its just looking at the bars being complete and not addressing the underlying calculations that you are performing.

Hope that helps

 

[Spin]

@Spin Also set your isComplete and setComplete statement in the loops to address the moving average series you are calculating.. otherwise its just looking at the bars being complete and not addressing the underlying calculations that you are performing.

Hope that helps

I'm afraid I'm not following now. You mean you want me to include code to tell the loops to stop calculating the values ?
Like so ??

for(int i = HMAperiod; i < series.size(); i++)
          { debug("Inside HMA 1 " + "index = " + index);
              if (series.isComplete(i)) continue;
              Double ma1 = series.ma(HMAmethod, i, HMAperiod/2, input);
              if (ma1 == null) continue;
              ma1 = ma1*2;
              Double ma2 = series.ma(HMAmethod, i, HMAperiod, input);
              if (ma2 == null) continue;
              series.setDouble(i, Values.HMA_Tmp, ma1 - ma2);
              series.setComplete(i, ma1);
              series.setComplete(i, ma2);
            }

That spits out the same result (hundreds of times that debug statement, only HMA drawn). I also tried with Values.HMA, but still not good.

Again: thanks for your time and allowing me to pick your brain !

 

[lndshrk]

Try something like this.. I also added a new variable and cast the HMAperiod to an integer for your moving average.

int latest = series.size()-1;

for(int i = HMAperiod; i <= latest; i++)
          { debug("Inside HMA 1 " + "index = " + index);  //this will just show you the index, not i
              if (series.isComplete(i, Values.HMA_Tmp)) continue;  //done calculating HHMA1?

              int HMAhalfperiod = (int) HMAperiod/2; 
              Double ma1 = series.ma(HMAmethod, i, HMAhalfperiod, input);
              if (ma1 == null) continue;
              ma1 = ma1*2;
              Double ma2 = series.ma(HMAmethod, i, HMAperiod, input);
              if (ma2 == null) continue;

              debug("line #  i => " + i + " ma1 => " + ma1 + " ma2 => " + ma2); //are you getting values here?
              series.setDouble(i, Values.HMA_Tmp, ma1 - ma2);
              series.setComplete(i, Values.HMA_Tmp);
            
            }

also be sure that you have a proper declaration in after your RuntimeDescriptor so it will draw..
Something along the lines of
desc.declarePath( Values.HMA_tmp, Inputs.Path);

 

[Spin]

I got it all working within 'Calculate' now !! ??
It's quite slow when dropping it onto a new chart, but it's also providing me with nice signals.

Thanks a ton @lndshrk ?

 

[jibanes]

@Spin could you please share the final code?

 

[Spin]

@jibanes please just tell me what it is exactly you are trying to achieve.

I will then see if I can help you out

For now I will tell you this already:
I use 'Calculate' for checks / calculations that need to be done on EVERY bar of a dataseries (e.g.: drawing a MA or another path, or a 'crossed above/below a certain value') and I use CalculateValues for checks / calculations that only need to be performed on a SELECTION of bars in a series (e.g.: for the last 20 bars, did this or this occur?)

 

[jibanes]

@Spin Thank you.

Few questions: can you use calculateValues and still be able to create a strategy relying on this? Were you able to _entirely_ replace MW's calculateValues and emulate the same behavior? Including when "supportsBarUpdates" is set to true? Reasons why I'm interested doing that is because in *some* cases, I can group computations (per bar) to save time.

 

[Spin]

@jibanes You sure can create Studies that rely solely on 'CalculateValues'. Look into the 270+ Studies that the good people of MW provide for free: there are many that do not use 'Calculate'.

You could off course also write your own classes and -using the building blocks that MW's SDK provides- emulate similar behavior.
I am not sure about 'SupportsBarUpdates', but you could easily figure that out for yourself: create a simple 'skeleton'-class that holds all the calculations that you want per bar in onCalculate, and put debugstatements right after every calculation. If everything outputs nicely to the Study log, you have your answer.

 

[jibanes]

But I believe those studies that use "calculateValues()" as opposed to "calculate()" aren't able to run strategies, is that correct? Because then again, a strategy works "step-by-step" and my guess is that "calculateValues()" was implemented _before_ MW had the strategy feature. If you want to implement a strategy (on a study); then you MUST use "calculate()" (I'm not even sure a strategy would even call "calculateValues()" all together).

 

[Spin]

Ah, that is another good question

I will honestly admit that I have not encountered such a problem, so I have no clue

You might be right when it comes to Strategies not calling 'calculateValues' all together. But then again: can you not MAKE them call it ?

Anyone else would like to pitch in ? @MotiveWave_Joe perhaps ?

 

[jibanes]

Because it would be extra computation to apply the entire algorithm to the entire dataset, by that I mean calculate() is meant to proceed step-by-step, so it's meant to be used for strategies and/or replay(s). Perhaps Joe can confirm?

 

[MotiveWave_Joe]

Hi,
I can check with development as to whether or not studies that use calculateValues() can be used in strategies. From what I see in the documentation, it seems that calculate method is called by calculateValues method for every bar in the data series and by default the calculateValues method is called on events where the data series has been affected.

 

[jibanes]

@motivewave_joe> I can check with development as to whether or not studies that use calculateValues() can be used in strategies.

Thanks, please let us know.

 

[gringojoey]

any update?

Hi,
I can check with development as to whether or not studies that use calculateValues() can be used in strategies. From what I see in the documentation, it seems that calculate method is called by calculateValues method for every bar in the data series and by default the calculateValues method is called on events where the data series has been affected.
View attachment 399