Wednesday, November 18, 2015

Crons in Maximo

When a Cron is Scheduled

When the CRONTASKINSTANCE is changed, it is made active or the schedule is modified, the new values are saved to the database and the RELOADREQTIME field is updated with the current time.

In each Maximo server, a background thread, called the Cron Monitor Thread runs.  Every 60 seconds (or how many seconds are defined in mxe.cronTaskMonitorInterval property) it looks at CRONTASKINSTANCE for all records with a RELOADREQTIME greater than the last time it checked.  If the cron isn't running and the instance is marked ACTIVE=1, then a new cron task thread is started for the cron. If the cron is running and the instance is marked ACTIVE=0, then the cron thread is woken and told to shutdown. If the cron is running and the instance is marked ACTIVE=1, then the thread is woken and told that an update is pending.  If Maximo is in ADMIN MODE, the cron monitor thread doesn't do anything.

When a new cron thread is started, it goes through these steps:

  1. A new Java Thread is started
  2. sleep 60 seconds, or how many seconds are defined in mxe.cronTaskInitDelay
  3. call the cron's init() method
  4. insert a new record in TASKSCHEDULER table if needed
  5. get last run information from TASKSCHEDULER table.
  6. calculate the next run time
  7. call the cron's start() method.
  8. loop and sleep until the next scheduled date arrives
  9. when the next scheduled date arrives
    if the last cron run was less than 300 minutes ago, only the maximo server that executed the cron last time can run the cron now, update TASKSCHEDULER with latest run information.
    if the last cron run was more than 300 minutes ago, this server can claim the cron and run it.  Update TASKSCHEDULER with the latest run information
  10. call the cron's action() method.


Cron Shutdown follows these steps:

  1. Cron task is marked for shutdown
  2. Thread is woken up.
  3. call the cron's stop() method.
  4. call the cron's shutdown() method.

These steps occur within each maximo server that can run crons.

Why do I have to schedule tasks to run in the future if I want to run them now?

When the Reload Request action is run, it updates the RELOADREQTIME on the CRONTASKINSTANCE record.  When the Cron Monitor Thread run's, it sets a pendingUpdate flag and attempts to wake up the thread.  Depending on where it's sleeping when it is awoken, it can slip into the action() branch before it realizes that the schedule has changed.  If the cron is already running when the pending flag is set, it might see the flag and act on it or it might not depending on when the flag is set.

The 60 second monitor interval plus the 60 second pause before intialization makes an absolute lower bound of at least two minutes before the next run time is calculated.  The longer the cron's init method takes to execute, that will push out the amount of time in advance the cron must be scheduled.  The number of cron nodes will also impact the amount of time in advance to schedule the crons.  Cron threads are started on each cron node in the cluster.  There can be database contention on TASKSCHEDULER that will add an additional amount of time before the next run date is calculated in step 6.

Finally, there is also clock skew to consider.  The person entering the scheduled time is either looking at their computer time, watch or cell phone.  Maximo schedules based on the time on the database server.  If the database machine is running faster than the user's clock, then after all the delays mentioned earlier, the scheduled time will have passed by the time it comes to determine the next run time.



1 comment: