Seems that the following line in the config.php didn't work at the default. Is it possible that it's actually miliseconds and not seconds?
// The number of seconds the operator has to accept chat before
// it polls to next operator
$conf['session_timeout'] = '30';
I had to increase it to 1800 for it to work for me.