Guides

Training

Data layout, batching and reading the training log.

Data layout

Both train_x and train_y are flat lists. For a network with 2 inputs and 3 outputs and N samples, train_x has 2N numbers and train_y has 3N.

python

# samples: (a1, a2), (b1, b2)
train_x = [a1, a2,  b1, b2]

If the lengths do not fit, fit raises a ValueError that states the expected length.

Batching

batch samples are processed, their gradients accumulated, then the weights are updated once with the gradient averaged over the batch. batch=1 is plain SGD. Larger batches give smoother but fewer updates per epoch.

Samples are visited in order and not shuffled between epochs. If your data is sorted by class, shuffle it before calling fit.

The training log

fit prints a progress bar for every epoch, and a loss line whenever epoch % log_step == 0. Use a larger log_step for quieter output. Do not pass 0, which would divide by zero in the C library.

Plotting the loss

The Python wrapper does not return the loss. fit only prints it. To plot a loss curve in your own code, train one epoch at a time and measure it yourself:

python

import matplotlib.pyplot as plt

history = []
for _ in range(60):
    model.fit(x, y, lr=0.1, epochs=1, batch=16, log_step=1)
    err = sum((model.forward(x[i*2:i*2+2])[0] - y[i]) ** 2 for i in range(len(y)))
    history.append(err / len(y))

plt.plot(history)
plt.show()

On the playground, Matplotlib works out of the box and figures appear under the output.

Tuning tips

If the loss turns to NaN, lower lr.
If it plateaus high, add neurons or another hidden layer.
Inputs scaled to roughly −1 to 1 train noticeably faster.
Values are stored as 32-bit floats, so expect float32 precision.

Losses

Saving and loading