Hey guys
Today we'll be taking a look at the variation of lambda parameter( how far we look ahead)
and seeing which eligiblity trace methods performs best for which ranges of lambda.
*Matlab code for reproducing plots*
Today we'll be taking a look at the variation of lambda parameter( how far we look ahead)
and seeing which eligiblity trace methods performs best for which ranges of lambda.
*Matlab code for reproducing plots*
As you guys might be aware, lambda=1 corresponds to no bootstraping case and lambda=0 performs bootstraping after 1 step transition.
According to Rich Sutton, Dutch traces are currently state of the art in terms of performance.
This is fortified by the plots, at low values of lambda, dutch traces perform as good as any others.
However, an interesting thing to note is that at high lambda value, replacing traces performs best.This may or may not be specific to this domain, however this was consistently noticed across many runs of the algorithm.
Intermediate values of lambda are often the way to go when generally coming across a new problem, however in this case using different type of traces slightly changed the convergence rates.
Feel free to tweak the parameters in the upload code and let me know if you find anything else intersting from the plots.
One thing that was however clear with all values was that non zero lambda showed faster convergence than lambda=0, the pure bootstrapping case.
Cheers.
