Initially you should try starting/stopping the celery service using:
$ /etc/init.d/columbus-celeryd (stop/start/restart/status)
If the celery service is 'hung' and does not respond to either /etc/init.d/columbus stop, or $ /etc/init.d/columbus-celeryd stop then it might be necessary to kill the Celery related processes. Typically you will see more than 1 process. There's a master but depending on the activity there will be several worker processes. They all run under the columbus account e.g:
$ ps f -fU columbus
UID PID PPID C STIME TTY STAT TIME CMD
columbus 12688 12687 0 08:01 ? S 0:01 nginx: worker process
columbus 12782 1 0 08:01 ? Sl 3:11 /usr/local/PerkinElmerCTG/Columbus2.8/webapp/virtualenv/bin/python -m celery.__main__ worker --app=columbus -n c
columbus 9627 12782 2 13:25 ? Sl 0:01 \_ /usr/local/PerkinElmerCTG/Columbus2.8/webapp/virtualenv/bin/python -m celery.__main__ worker --app=columbus
columbus 9731 12782 0 13:25 ? Sl 0:00 \_ /usr/local/PerkinElmerCTG/Columbus2.8/webapp/virtualenv/bin/python -m celery.__main__ worker --app=columbus
columbus 9751 12782 3 13:25 ? Sl 0:01 \_ /usr/local/PerkinElmerCTG/Columbus2.8/webapp/virtualenv/bin/python -m celery.__main__ worker --app=columbus
You'd need to kill the master with PID 12782 but before that the slave processes must be killed, you can use pkill -P with the parent id to do so:
$ sudo kill -9 -P 12782
# Now kill the master
$ sudo kill -9 12782
Now if you start Celery again it should bring it back to life. If this doesn't help then there is potential for something to be broken meaning that the image rendering jobs will fail, if that is the case then check the /var/log/columbus/web/columbus-images-service.log file for errors and forward on to the PKI informatics support team for further troubleshooting advice.
RAH
↧