Wednesday, November 18, 2015

Querying Using Non-Persistent Fields

I recently ran into a situation where I wanted to make a non-persistent field searchable in a table filter. The requirement was to make a table searchable based on whether a complex condition was true or false.

First, I created a non-persistent field on the table through Database Configuration.

Next, I added the field to my table and made it filterable. This was done by adding filterable="true" to the tablecol definition.

<tablecol dataattribute="CID_DEFECT" id="cid_defect" filterable="true" inputmode="readonly"/>


Normally, non-persistent fields are not filterable. I created the following Java class to make non-persistent fields work in a table filter.  The package space is important because it requires replacing the value of a package protected field.

package psdi.mbo;

import java.rmi.RemoteException;
import java.util.HashMap;
import java.util.Locale;
import java.util.Map;
import java.util.TimeZone;

import psdi.security.UserInfo;
import psdi.util.MXException;

/**
 * An MboQuery object that will accept some non-persistent fields without
 * adding them to the where clause.  This is to permit a non-persistent field
 * in a table filter row where the sql logic is implemented in the databean.
 *
 * @author Martin Nichol
 *
 */
public class NPMboQbe extends MboQbe {

    /** the list of permitted non-persistent attributes and their values.  */
    private HashMap<String, String> permittedAttributes = new HashMap<String, String>();

    /**
     * Constructor.
     *
     * @param ms mbosetinfo of the owning mboset.
     * @param processML the processML flag of the owning mboset.
     * @param userInfo the user who owns the owning mboset
     * @param npAttributes the non-persistent attributes to permit.
     */
    public NPMboQbe(MboSetInfo ms, boolean processML, UserInfo userInfo, String[] npAttributes) {
        super(ms, processML, userInfo);
        initializeNpFields(npAttributes);
    }

    /**
     * @param ms mbosetinfo of the owning mboset.
     * @param l the user's locale.
     * @param tz the user's timezone.
     * @param ml the processML flag of the owning mboset.
     * @param userInfo the user who owns the owning mboset.
     * @param npAttributes the non-persistent attributes to permit.
     */
    public NPMboQbe(MboSetInfo ms, Locale l, TimeZone tz, boolean ml, UserInfo userInfo, String[] npAttributes) {
        super(ms, l, tz, ml, userInfo);
        initializeNpFields(npAttributes);
    }


    @Override
    public void setQbe(String attr, String expr) throws MXException {
        if (permittedAttributes.containsKey(attr)) {
            permittedAttributes.put(attr, expr);
        } else {
            super.setQbe(attr, expr);
        }
    }

    @Override
    public String getQbe(String attrStr) throws MXException {
        if (permittedAttributes.containsKey(attrStr)) {
            return permittedAttributes.get(attrStr);
        } else {
            return super.getQbe(attrStr);
        }
    }

    @Override
    public void resetQbe() {
        super.resetQbe();
        for (Map.Entry me : permittedAttributes.entrySet()) {
            me.setValue("");
        }
    }

    /**
     * From a list of attributes, initialize the map of values.
     * @param npAttributes the list of attributes to allow.
     */
    private void initializeNpFields(String[] npAttributes) {
        for (String attr : npAttributes) {
            permittedAttributes.put(attr, "");
        }
    }

    /**
     * Associate an NPMboQbe with a given MboSet.
     * @param msr the mboset to replace the MboQbe object.
     * @param npAttributes the list of attributes to allow.
     * @throws RemoteException if an RMI problem occurs.
     * @throws MXException if a Maximo problem occurs.
     */
    public static void assignNewQbe(MboSetRemote msr, String[] npAttributes) throws RemoteException, MXException {
        MboSet ms = (MboSet)msr;
        MboQbe qbe = new NPMboQbe(ms.getMboSetInfo(), ms.getClientLocale(), ms.getClientTimeZone(), ms.processML(), ms.getUserInfo(), npAttributes);
        ms.qbe = qbe;
    }
}


Then I extended the DataBean associated with the table to apply the sql driven by the non-persistent field. If a DataBean isn't defined for the table, create it and assign it.

@Override
public synchronized void setQbe(String attribute, String expression) throws MXException {
    super.setQbe(attribute, expression);
    if ("CID_DEFECT".equals(attribute)) {
       if (expression.contains("Y")) {
          getMboSet().setWhere("sql expression for condition true");
       } else if (expression.contains("N")) {
          getMboSet().setWhere("sql expression for condition false");
       } else {
          getMboSet().setWhere("");
       }
    }
}

@Override
protected void initialize() throws MXException, RemoteException {
    super.initialize();
    NPMboQbe.assignNewQbe(getMboSet(), new String[] { "CID_DEFECT" });
}

The non-persistent field can now affect the results displayed in the table.

Crons in Maximo

When a Cron is Scheduled

When the CRONTASKINSTANCE is changed, it is made active or the schedule is modified, the new values are saved to the database and the RELOADREQTIME field is updated with the current time.

In each Maximo server, a background thread, called the Cron Monitor Thread runs.  Every 60 seconds (or how many seconds are defined in mxe.cronTaskMonitorInterval property) it looks at CRONTASKINSTANCE for all records with a RELOADREQTIME greater than the last time it checked.  If the cron isn't running and the instance is marked ACTIVE=1, then a new cron task thread is started for the cron. If the cron is running and the instance is marked ACTIVE=0, then the cron thread is woken and told to shutdown. If the cron is running and the instance is marked ACTIVE=1, then the thread is woken and told that an update is pending.  If Maximo is in ADMIN MODE, the cron monitor thread doesn't do anything.

When a new cron thread is started, it goes through these steps:

  1. A new Java Thread is started
  2. sleep 60 seconds, or how many seconds are defined in mxe.cronTaskInitDelay
  3. call the cron's init() method
  4. insert a new record in TASKSCHEDULER table if needed
  5. get last run information from TASKSCHEDULER table.
  6. calculate the next run time
  7. call the cron's start() method.
  8. loop and sleep until the next scheduled date arrives
  9. when the next scheduled date arrives
    if the last cron run was less than 300 minutes ago, only the maximo server that executed the cron last time can run the cron now, update TASKSCHEDULER with latest run information.
    if the last cron run was more than 300 minutes ago, this server can claim the cron and run it.  Update TASKSCHEDULER with the latest run information
  10. call the cron's action() method.


Cron Shutdown follows these steps:

  1. Cron task is marked for shutdown
  2. Thread is woken up.
  3. call the cron's stop() method.
  4. call the cron's shutdown() method.

These steps occur within each maximo server that can run crons.

Why do I have to schedule tasks to run in the future if I want to run them now?

When the Reload Request action is run, it updates the RELOADREQTIME on the CRONTASKINSTANCE record.  When the Cron Monitor Thread run's, it sets a pendingUpdate flag and attempts to wake up the thread.  Depending on where it's sleeping when it is awoken, it can slip into the action() branch before it realizes that the schedule has changed.  If the cron is already running when the pending flag is set, it might see the flag and act on it or it might not depending on when the flag is set.

The 60 second monitor interval plus the 60 second pause before intialization makes an absolute lower bound of at least two minutes before the next run time is calculated.  The longer the cron's init method takes to execute, that will push out the amount of time in advance the cron must be scheduled.  The number of cron nodes will also impact the amount of time in advance to schedule the crons.  Cron threads are started on each cron node in the cluster.  There can be database contention on TASKSCHEDULER that will add an additional amount of time before the next run date is calculated in step 6.

Finally, there is also clock skew to consider.  The person entering the scheduled time is either looking at their computer time, watch or cell phone.  Maximo schedules based on the time on the database server.  If the database machine is running faster than the user's clock, then after all the delays mentioned earlier, the scheduled time will have passed by the time it comes to determine the next run time.



Thursday, August 13, 2015

MXJUnit: Testing Crons

MXJUnit provides the ability to unit test crons.  This means that, within a JUnit test case, it is possible to execute a cron and test what it does.  MXJunit goes one step further and allows you to execute the cron with a subclass of what's defined in Maximo.  This allows a change to change the behaviour of the cron for the test.  Normally, I use this to provide a different SQL where clause so the test operates on a smaller and/or known subset of the data.

Most MXJUnit tests will extend from MaximoTestHarness.  A test that has to execute a cron task must extend from CronTaskTestHarness.   This provides methods to execute a cron, but also performs some additional work behind the scenes that isn't normally required.

This example executes the KPIHISTORY cron which goes through the KPIHISTORY table and deletes records more than a year old.  These examples should be executable in any Maximo environment.


 public class CronTaskDemo1Test extends CronTaskTestHarness {  
      /**  
       * Test that KPIHISTORY cron will delete old KPIHISOTRY records.  
       *   
       * @throws RemoteException if an RMI problem occurs.  
       * @throws MXException if a Maximo problem occurs.  
       */  
      @Test  
      public void testKPIHistory() throws RemoteException, MXException {  
           // create a new KPI History record for testing.  
           MboRemote kpiHistory = createKpiHistory();  

           // load it and verify that it exists and is properly populated  
           kpiHistory = refresh(kpiHistory);  
           assertNotNull(kpiHistory);  
           assertEquals(pastDate(), kpiHistory.getDate("RECORDEDON"));  

           // run the cron  
           runCron("KPIHISTORY", "KPIHISTORY1YEAR");  

           // attempt to reload the KPIHISTORY and note that it is gone.   
           kpiHistory = refresh(kpiHistory);  
           assertNull(kpiHistory);  
      }

     // ... snip ...  
 }  

The full source is available in CronTaskDemo1Test.java

The runCron() method executes the cron.  It must refer to an existing definition in the Maximo cron task tables (CRONTASKDEF, CRONTASKINSTANCE).  It will execute the cron with the parameters defined for the named cron task instance.  The cron does not need to be active in Maximo for the cron to execute it.

That's all there is to running a cron in an MXJUnit test.  After the test, make sure to test that the cron performed the intended work.


 public class CronTaskDemo2Test extends CronTaskTestHarness {  
   
      public static class DemoCron extends KPIHistoryCleanupCronTask {  
           @Override  
           public void cronAction() {  
                throw new RuntimeException("Stopped!");  
           }            
      }  
        
      /**  
       * Test that KPIHISTORY cron will delete old KPIHISOTRY records.  
       *   
       * @throws RemoteException if an RMI problem occurs.  
       * @throws MXException if a Maximo problem occurs.  
       */  
      @Test  
      public void testKPIHistory() throws RemoteException, MXException {  
           try {  
                // run the cron  
                runCron(DemoCron.class, "KPIHISTORY", "KPIHISTORY1YEAR");  
                fail("Exception expected");  
           } catch (RuntimeException e) {  
                assertEquals("Stopped!", e.getMessage());  
           }       
      }            
 }  

The full source is available in CronTaskDemo2Test.java

Many crons I've written involve data transfers or data updates.  By writing the cron with a method to retrieve or generate an SQL where clause to identify the data on which to operate, it is possible to use MXJUnit to substitute a different where clause for testing.

This example replaces the cronAction() of the KPIHISTORY cron to throw an error.  A new class, DemoCron is created to extend from KPIHistoryCleanupCronTask.  The cronAction() method is overridden to perform a different action.  The cron is executed using the alternative runCron() that accepts a Class object.  It is that Class that will be instantiated instead of the one defined in the CRONTASKDEF table.


      public static class DemoCron extends PMWoGenCronTask {  
           @Override  
           public void cronAction() {  
                throw new RuntimeException("Stopped!");  
           }            
      } 

The full source is available in CronTaskDemo3Test.java

As a safety precaution,  the CronTaskTestHarness makes sure that the class passed in the first parameter is a subclass of the class defined in CRONTASKDEF.

If the DemoCron is changed to extend from PMWoGenCronTask, the test will fail with a message that "Test class must be a subclass of com.ibm.tivoli.maximo.report.kpi.KPIHistoryCleanupCronTask."  This will catch situations where the cron class has changed in the database or the developer makes a mistake in the class definition.

Dynamically Adding a Transactable

I have run into several situations where I have to work on data that could have been modified through one of several different relationships.  It's not practical to consider each possible relationship.  What I have found that works, is to dynamically add a Transactable object.

Here is an example of adding the Transactable object.

 public class CustomMeasurement implements CustomMeasurementRemote {  
   /* ... snip ... */
   @Override
   public void save() throws MXException, RemoteException {
      if (/* condition true */) {
         getMXTransaction().add(new CustomTransactable(this));
      }
      super.save();
   }

   @Override
   public void customWork() throws MXException, RemoteException {
      // perform work that
   }
}


In this example, the custom Measurement object overrides the save method.  When it is determined that the customWork() should be performed, the CustomTransactable is added to the current transaction.  In this example, it's done in the save() method, but it can be done in any method, at any time.

The CustomTransactable looks like this:

 public class CustomTransactable implements Transactable {  
   
   private CustomMeasurement mbo;  
   
   public CustomTransactable(CustomMeasurement mbo) {  
     super();  
     this.mbo = mbo;  
   }  
   
   @Override  
   public void commitTransaction(MXTransaction arg0) throws MXException, RemoteException {  
     return;  
   }  
   
   @Override  
   public void fireEventsAfterDB(MXTransaction arg0) throws MXException, RemoteException {  
     return;  
   }  
   
   @Override  
   public void fireEventsAfterDBCommit(MXTransaction arg0) throws MXException, RemoteException {  
     MboSetRemote msr = mbo.getMboServer().getMboSet("MEASUREMENT", mbo.getUserInfo());  
     CustomMeasurementRemote mr = (CustomMeasurementRemote)msr.getMboForUniqueId(mbo.getUniqueIDValue());  
     mr.customWork();  
     msr.save();  
   }  
   
   @Override  
   public void fireEventsBeforeDB(MXTransaction arg0) throws MXException, RemoteException {  
     return;  
   }  
   
   @Override  
   public void rollbackTransaction(MXTransaction arg0) throws MXException, RemoteException {  
     return;  
   }  
   
   @Override  
   public void saveTransaction(MXTransaction arg0) throws MXException, RemoteException {  
     return;  
   }  
   
   @Override  
   public void undoTransaction(MXTransaction arg0) throws MXException, RemoteException {  
     return;  
   }  
   
   @Override  
   public boolean validateTransaction(MXTransaction arg0) throws MXException, RemoteException {  
     return true;  
   }  
 }  

The constructor takes a reference to the mbo that will be updated.

All the methods, with two exceptions, have empty stubs, but they need to be defined.

The first exception is validateTransaction() returns true.

The second exception is fireEventsAfterDBCommit().  This method is called have the data has been committed to the database.  In this example, it loads a fresh copy of the mbo from the database, calls a method to act on it, and then saves the changes back to the database.

This occurs within the same thread as the call to MboSet.save().  The call will not return until after the CustomTransactable has run.

Since the code executes after the commit, the changes will appear in a separate EAudit record, if the mbo is EAudit enabled.

Getting back to my original problem.  The code in the Mbo or in the screens updates data through one of several different relationships.  It's not easy to tell through which one in the code.  By adding the Transactable object to operate on the object after the commit, the code can use whichever relationship is most appropriate to work on the data.


Monday, June 29, 2015

Unexpected MboSet Reset

I spent about a half day on this one.

We have a customization where the MEASUREMENT table contains a TICKETID column pointing to a record in the TICKET table.

There is a relationship called TICKETS with these properties:
PARENT: MEASUREMENT
CHILD: TICKET
WHERECLAUSE: ticketid = :ticketid


 MboRemoteSet measurements = /* an mboset of measurements. */  
   
 MboRemote measurement = /* a measurement record being updated */  
 MboSetRemote tickets = measurement.getMboSet("TICKETS");  
 MboRemote ticket = tickets.add();  
 measurement.setValue("TICKETID", ticket.getString("TICKETID"));  
   
 /* in a later block of code, but in the same transaction */  
 MboSetRemote tickets = measurement.getMboSet("TICKETS");  
 MboRemote ticket = tickets.getMbo(0);  
 /* ticket is now null! */  

The problem was a reset() on the second getMboSet("TICKETS").

The first getMboSet("TICKETS") created a related MboSet to the TICKETS table.  Internally, Maximo stores the relationship where clause used to create that relationship.  In the first case, that relationship is "ticketid = ''".

In the second getMboSet("TICKETS") the relationship where clause has become "ticketid = 'new ticketid'".  Maximo sees this difference and resets the MboSet before returning it to the caller.

I ended up with a MEASUREMENT record referencing a TICKET that didn't exist in the database. 


The fix was to reset the relationship after setting TICKETID.

 measurement.setValue("TICKETID", ticket.getString("TICKETID"));  
   
 RelationInfo ri = measurements.getMboSetInfo().getRelationInfo(tickets.getRelationName()); 
 SqlFormat sf = new SqlFormat(measurement, ri.getSqlExpr()); 
 tickets.setRelationship(sf.format());  

This reset the relationship where clause between MEASUREMENT and TICKETS without resetting the MboSet and losing my new TICKET object.

Tuesday, June 23, 2015

MXJUnit: Introduction

MXJUnit is a unit testing framework for Maximo by Interloc Solutions built on top of JUnit.  It permits testing of classes that normally run in the MXServer --- not UI classes like DataBeans --- within Eclipse itself.  This includes MBOs and field validators, but also crons and the MIF.


MXJUnit provides the following features:
  • capture e-mails from being sent during tests, but also permits writing tests to confirm that an e-mail has been sent and verify its contents.
  • capture logger output.
  • generate work orders.
  • run crons from a test, optionally using a subclass specific to the test.  This allows changing behaviour for the test, usually to have the cron work on a known subset of data.
  • capture output from PublishChannels.
  • send data to an Enterprise Service to test UserExits.
  • send data to an Object Structure to test processing classes.
  • replace Maximo property (MAXPROP) values, system properties and MAXVARS temporarily for the duration of a test.
  • replace an MBO class for the duration of a test.
  • automatically delete MBOs created during the test after the test has completed.

Writing test cases using MXJUnit should follow normal best practices for writing regular JUnit tests.


MXJUnit tests should extend from MaximoTestHarness or one of its subclasses.  Special subclasses exist for testing Crons and for testing MIF.  MaximoTestHarness provides methods for interacting with Maximo.  Some of the more useful methods are:
  • generateWorkOrder:  generates work orders given a PM object and lead time information.
  • getLogCapture: get the logger output generated during the test.
  • getMboSet: get an MboSet, optionally specifying a user and app to enable application security.
  • refresh: given an MBO, reload a fresh copy from the database.
  • refreshUserSecurity: after making ApplicationAuth changes, reloads security settings for a given user.
  • spyOnMXServer: return an MXServer instance that has been mocked using Mockito.

Here is a sample unit test that retrieves a work order by wonum and verifies that the record returned was the record expected.

 package com.interlocsolutions.demo;   
 import java.rmi.RemoteException;   
 import static org.junit.Assert.*;   
 import org.junit.Test;   
 import psdi.mbo.MboRemote;   
 import psdi.mbo.MboSetRemote;   
 import psdi.util.MXException;   
 import com.interlocsolutions.maximo.junit.MaximoTestHarness;   
   
 public class MXDemoTest extends MaximoTestHarness {     
   @Test     
   public void testWorkOrder() throws MXException, RemoteException {       
    // Setup       
    MboSetRemote msr = getMboSet("WORKORDER");       
    msr.setWhere("wonum = '1022'");       
    MboRemote mr = msr.getMbo(0);       
   
    // Execute       
    String wonum = mr.getString("WONUM");           
   
    // Verify       
    assertEquals("1022", wonum);     
   }   
 }  

In the Setup phase, a WORKORDER MboSet is created and the work order with wonum='1022' is retrieved.

In the Execute phase, the WONUM field is retrieved.

In the Verify phase, the WONUM is confirmed to be 1022.  Every test should assert something or the test doesn't test anything.


Here is a sample walk through creating and running an MXJUnit test.


Hello

I've been working with Maximo for over 10 years.  I started with version 5.

What I'm hoping to do is provide some deep technical information on Maximo.  These would be things I've discovered over time and usually out of necessity.