This optimizer performs an implicit stochastic gradient descent
algorithm. It is mainly used within a `bamlss`

call.

```
opt_isgd(x, y, family, weights = NULL, offset = NULL,
gammaFun = function(i) 1/(1 + i), shuffle = TRUE,
CFun = function(beta) diag(length(beta)),
start = NULL, i.state = 0)
```

## Arguments

- x
For function `boost()`

the `x`

list, as returned from
function `bamlss.frame`

, holding all model matrices
and other information that is used for fitting the model.

- y
The model response, as returned from function
`bamlss.frame`

.

- family
A bamlss family object, see
`family.bamlss`

.

- weights
Prior weights on the data, as returned from function `bamlss.frame`

.

- offset
Can be used to supply model offsets for use in fitting,
returned from function `bamlss.frame`

.

- gammaFun
Function specifying the step length.

- shuffle
Should the data be shuffled?

- CFun
Hessian approximating function.

- start
Vector of starting values.

- i.state
Added to `gammaFUN()`

.

## Value

For function `opt_isgd()`

a list containing the following objects:

- fitted.values
A named list of the fitted values based on the last iteration
of the modeled parameters of the selected distribution.

- parameters
A matrix, each row corresponds to the parameter values of one iteration.

- sgd.summary
The summary of the stochastic gradient descent algorithm which can be printed and plotted.

## References

Toulis, P and Airoldi, EM (2015): Scalable estimation strategies based on
stochastic approximations: Classical results and new insights.
Statistics and Computing, 25, no. 4, 781--795.
doi: 10.1007/s11222-015-9560-y

## Note

Motivated by the lecture 'Regression modelling with large data sets'
given by Ioannis Kosmidis in Innsbruck, January 2017.

## Warning

CAUTION: Arguments `weights`

and `offset`

are not implemented yet!

## Examples

```
if (FALSE) {
set.seed(111)
d <- GAMart(n = 10000)
f <- num ~ s(x1) + s(x2) + s(x3) + te(lon, lat)
b <- bamlss(f, data = d, optimizer = opt_isgd, sampler = FALSE)
plot(b, ask = F)
## loop over observations a 2nd time
b <- bamlss(f, data = d, optimizer = opt_isgd, sampler = FALSE, start = parameters(b),
i.state = b$model.stats$optimizer$sgd.summary$i.state)
plot(b, ask = F)
## try differeent gammaFuns, e.g.,
# gammaFun <- function(i) .3/sqrt((1+i)) + 0.001
## testing some families
f2 <- bin ~ s(x1) + s(x2) + s(x3) + te(lon, lat)
b2 <- bamlss(f2, data = d, optimizer = opt_isgd, sampler = FALSE, family = "binomial")
f3 <- cens ~ s(x1) + s(x2) + s(x3) + te(lon, lat)
b3 <- bamlss(f3, data = d, optimizer = opt_isgd, sampler = FALSE, family = "cnorm")
}
```