ALDSP Null Pointer Exceptions

I have seen ALDSP null pointer exceptions a few times, with stack traces like this:

Caused by: java.lang.NullPointerException
        at com.bea.ld.dsmediator.update.DataServiceMediator.populateBindingTree(DataServiceMediator.java:2183)
        at com.bea.ld.dsmediator.update.DataServiceMediator.populateBindingTree(DataServiceMediator.java:2230)
        at com.bea.ld.dsmediator.update.DataServiceMediator.populateBindingTree(DataServiceMediator.java:2230)
        at com.bea.ld.dsmediator.update.DataServiceMediator.parseDataGraph(DataServiceMediator.java:1990)
        at com.bea.ld.dsmediator.update.DataServiceMediator.getUpdatePlan(DataServiceMediator.java:593)
        at com.bea.ld.dsmediator.update.DataServiceMediator.submit(DataServiceMediator.java:509)

Today it arose when I tried to submit an update to a DTO that showed a parent-child hierarchy.  I didn't seriously think that the update would work, but I don't think that an NPE is a particularly helpful error message.  I looked at the source of DataServiceMediator#populateBindingTree, but it was hard to identify the errant line.

ALDSP Update Order

In ALDSP 2.0, the manual states that when updating, containers are processed prior to contained objects, except when deleting when the reverse is true (see the Update Order section of the Client Application Developer's Guide).  It is interesting that the 2.1 version of the same manual makes no such claim.  Indeed, there is a reference in the 2.1 release notes to a bug (CR237186) in which update order is not respected.  It states that updates occur in alphabetical order, although it is not clear whether it is alphabetical by table name, data service name, or element name (the release notes weakly suggest that it might be element name).

Experimentation with 2.1 reveals that update order for insertions appears to be correct (container first) regardless of alphabetical order.  Deletions, on the other hand, do not appear to work properly.  I tried naming the elements in the DTO so that the container was alphabetically before the child, and vice versa.  Neither worked.

The 2.1 client guide suggests that an override function should be used to control update order.  I was unable to get even a trivial override function to work -- I would always get a NullPointerException.  Once I got even a single NPE, blanking out the override function in data service's property page would not restore functionality.  I would have to hand-edit the pragma to remove the override function XML element completely.

Incidentally, I couldn't get override functions to compile without adjusting the server classpath for the application to include wlsdo.jar from the liquidata/lib directory.  The manual does not mention this..

ALDSP Cascaded Deletes

Using ALDSP, there are problems if you try to delete an object that owns its children (and both are mapped into the same document).  Even though ALDSP inspects the foreign key constraints, it does not use this information to figure out cascaded deletes.  If you simply delete the parent object, you will get an exception like:

com.bea.ld.dsmediator.DataServiceException:
  java.sql.SQLException: [BEA][SQLServer JDBC Driver][SQLServer]
    DELETE statement conflicted with COLUMN REFERENCE constraint 'FK_ITEM_FACT'.
    The conflict occurred in database 'mmsandbox', table 'FACT', column 'ITEM_ID'.

To perform a cascaded delete successfully, you must first delete all of the child nodes from the data graph, then the parent node.  When you submit the altered data graph, it will do the right thing (in the right order, apparently, not in alphabetical order as claimed in the 2.1 release notes).

ALDSP vs. Multi-table Updates Again

After working throw the "Invalid SQL" problem, I returned to plan to try a parent-child insertion having renamed the child table so that it was alphabetically after the parent table.  No luck.  I also tried mapping the parent key field to the DTO in both the parent and child levels, hoping that that would make the shared nature of the key more apparent to the updater.  It didn't work.

The exception is:

java.rmi.RemoteException: EJB Exception: ; nested exception is: 
  com.bea.ld.dsmediator.DataServiceException:  Primary key attribute ITEM_ID on Object FACT is not provided.

ALDSP Invalid SQL Specified

I tracked down the cause of the ALDSP exception mentioned in yesterday's entry.  The stacktrace is exhibited below.  It is caused by specifying setting the 'auto number' property of a data service key to 'sequence' for anything other than Oracle or DB2.  In my case, I was using MS SQL Server but had accidentally selected 'sequence' instead of 'identity'.  Looking at the code for JDBCAdaptor.populateSequence, I saw that it calls SQLGenerator.getSQLSequenceFetch to generate an SQL statement to get the next sequence value.  The latter returns null for any database type other than Oracle or DB2.  The caller then tries to execute 'null' as the SQL statement which the JDBC driver catches as an error.  I suppose that one of the routines involved should have caught the mistake and thrown a more informative error message.

Here is the stacktrace:

[SQLServer JDBC Driver]Invalid SQL specified.
com.bea.ld.dsmediator.DataServiceException: java.sql.SQLException: [BEA][SQLServer JDBC Driver]Invalid SQL specified.
        at com.bea.ld.dsmediator.update.JDBCAdaptor.save(JDBCAdaptor.java:248)
        at com.bea.ld.dsmediator.update.DataServiceMediator.submit(DataServiceMediator.java:500)
        at com.bea.ld.dsmediator.update.DataServiceMediator.nestedSubmit(DataServiceMediator.java:1871)
        at com.bea.ld.dsmediator.update.DataServiceMediator.executeUpdatePlan(DataServiceMediator.java:1828)
        at com.bea.ld.dsmediator.update.DataServiceMediator.submit(DataServiceMediator.java:511)
        at com.bea.ld.dsmediator.update.DataServiceMediator.submit(DataServiceMediator.java:227)
        at com.bea.ld.ServerBean.submit(ServerBean.java:529)
        at com.bea.ld.Server_ydm4ie_EOImpl.submit(Server_ydm4ie_EOImpl.java:856)
        at com.bea.ld.Server_ydm4ie_EOImpl_WLSkel.invoke(Unknown Source)
        at weblogic.rmi.internal.BasicServerRef.invoke(BasicServerRef.java:492)
        at weblogic.rmi.cluster.ReplicaAwareServerRef.invoke(ReplicaAwareServerRef.java:108)
        at weblogic.rmi.internal.BasicServerRef$1.run(BasicServerRef.java:435)
        at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:363)
        at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:147)
        at weblogic.rmi.internal.BasicServerRef.handleRequest(BasicServerRef.java:430)
        at weblogic.rmi.internal.BasicExecuteRequest.execute(BasicExecuteRequest.java:35)
        at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:224)
        at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:183)
Caused by: java.sql.SQLException: [BEA][SQLServer JDBC Driver]Invalid SQL specified.
        at weblogic.jdbc.base.BaseExceptions.createException(Unknown Source)
        at weblogic.jdbc.base.BaseExceptions.getException(Unknown Source)
        at weblogic.jdbc.base.BaseStatement.commonValidateSQL(Unknown Source)
        at weblogic.jdbc.base.BaseStatement.execute(Unknown Source)
        at weblogic.jdbcx.base.BaseStatementWrapper.execute(Unknown Source)
        at weblogic.jdbc.wrapper.Statement.execute(Statement.java:386)
        at com.bea.ld.dsmediator.update.JDBCAdaptor.populateSequence(JDBCAdaptor.java:328)
        at com.bea.ld.dsmediator.update.JDBCAdaptor.save(JDBCAdaptor.java:113)
        ... 17 more

ALDSP Primary Key Not Provided

I got to the bottom of the exception "Primary key attribute ... on Object ... not provided".  It is caused when you insert boilerplate XML container elements in your DTO.  For example, if the document looks like this:

<Item id="1" type="1">
  <PropertyList>
    <Property id="1" value="2"/>
    ...

and the Item element is mapped to the parent table and the Property element is mapped to the child table, then you will get the exception.  However, if you delete the organizing element PropertyList:

<Item id="1" type="1">
  <Property id="1" value="2"/>
  ...

then the mapping works properly and updates will work.

ALDSP Recursive Type Definitions

I had a schema that defined a HierarchyList as a list of Hierarchy elements each containing a list of Item elements each containing (recursively) a list of Item elements.  When I tried to submit an update that added an empty Hierarchy element to an empty HierarchyList, I got a StackOverflowException (with no stack trace).  I guess ALDSP does not handle recursive schemas gracefully.

ALDSP vs. Multi-table Updates

I created a data service that reports on a couple of tables that have a parent-child relationship.  This was SQL Server, so the primary keys were auto-generating 'identity' keys.  If I updated the ALDSP document to create a new parent node and submit it, all is well.  But if I submit a document that creates both a parent and child node, then the update fails complaining that no value was supplied for the parent key in the child table.  "Aha!" I thought, I remember reading in the BEA documentation that updates occur in alphabetical order by table name and, as luck would have it, that was the wrong order for my tables.  So I renamed the child table to sort alphabetically after the parent table and tried again.

No success (yet).  I keep getting the exception::

java.sql.SQLException: [BEA][SQLServer JDBC Driver]Invalid SQL specified

I have turned all of the logging in ALDSP, WebLogic, and the relevant application to full-on maximum -- but ALDSP doesn't show the SQL in question.  In fact, I get this error even when I try to update the parent alone, so something else is broken.  I'll keep looking...

MS SQL Server Enterprise Manager vs. BEA WebLogic Workshop

I find that if I have the MS SQL Server Enterprise Manager open at the same time as BEA WebLogic Workshop, the latter feels much more clunky to use.  Frequently, there are long delays between the moment I type a key or click the mouse, and when the character appears or the click is processed.  As soon as I close Enterprise Manager, the behaviour goes back to normal.  I wonder if there is some adverse interaction, or if Enterprise Manager is interfering with everything else, Workshop or otherwise?  I might just be noticing it in Workshop since it is not exactly snappy at the best of times.

ALDSP vs. Renaming Tables

I renamed a table in an SQL table and then used Workshop's 'update metadata' function to try to synchronize an ALDSP project that used the renamed table.  So now I had two physical data services for the renamed table, one for the old name and one for the new name.  The new data service had none of the relationships, presumably because ALDSP was not smart enough to find the foreign keys that involve the existing tables (fair enough).  So, I tried some hand surgery to copy the relationships from the old data service into the new.  Bad idea.  ALDSP kept trying to second guess me and started modifying (and even commenting out!) data service functions all over the place.  It got so tangled up that my only recourse was to blow away all the physical data services and rebuild them from scratch.  Ouch.  Moral of the story -- when you rename a table, hand edit the corresponding physical data service to match.

Storing Original Units of Measure

I have been trying to track down the source of the legend that there exist jurisdictions in which regulations or laws demand that the original units of measure be stored in databases.  So far, all I have found is a passing reference to this fact in section 8.2 (units of measure, paragraph 8.2.3) of the PPDM Architectural Principles guide, version 3.7.

Submitting AquaLogic Documents

I discovered that when you build an ALDSP document that contains two related entities, you must take care when you map the join fields to the document.  You must map the primary key side of the relationship to the output document.  If you map the foreign key side, then you will get a runtime error message like:

com.bea.ld.dsmediator.DataServiceException: Primary key attribute PROPERTY_ID on Object FACT is not provided.

Actually, the rule may be that you have to map the key from the outer element (regardless of which side of the foreign key constraint it comes from).  More experimentation is needed.

Data Models

While noodling about how to create a credible metamodel for a particular model, I started considering whether a dimensional model might be a better choice than an entity-relationship model.  I thought that I would take this opportunity to capture links to relevant foundational papers:

• E.F. Codd, A Relational Model of Data for Large Shared Data Banks
• Peter P. Chen, The Entity-Relationship Model -- Toward a Unified View of Data
• Ralph Kimball, A Dimensional Modeling Manifesto

Python IDEs

PyScripter is a nice, free, IDE for Python.  PyDev seems to be a commonly used Eclipse plug-in, but it is switching to a commercial model with all the good features in the commercial PyDev Extensions product.

Ant, WebLogic 8.x, JDBC Connection Pools

WebLogic provides an Ant task called wlconfig that can be used to change lots of settings in the WLS configuration.  For example, you can create a JDBC connection pool and data source like this:

    <wlconfig url="t3://localhost:7001" username="weblogic" password="weblogic">

      <query domain="mydomain81" type="Server" name="myserver81" property="server-jmx-name"/>

      <query type="JDBCConnectionPool" name="ZotPool">
        <delete/>  <!-- does nothing if the pool does not exist -->
      </query>
      <create type="JDBCConnectionPool" name="ZotPool">
        <set attribute="DriverName" value="weblogic.jdbcx.oracle.OracleDataSource"/>
        <set attribute="URL" value="jdbc:bea:oracle://mm1:1521"/>
        <set attribute="Properties" value="serverName=mm1;SID=dev1;portNumber=1521;user=myuserid"/>
        <set attribute="Password" value="mypassword"/>
        <set attribute="ShrinkingEnabled" value="true"/>
        <set attribute="InitialCapacity" value="0"/>
        <set attribute="TestConnectionsOnReserve" value="true"/>
        <set attribute="TestTableName" value="SQL SELECT 1 FROM DUAL"/>
        <set attribute="Targets" value="${server-jmx-name}"/> <!-- must be the last attribute -->
      </create>

      <query type="JDBCTxDataSource" name="ZotDataSource">
        <delete/>  <!-- does nothing if the data source does not exist -->
      </query>
      <create type="JDBCTxDataSource" name="ZotDataSource">
        <set attribute="JNDIName" value="ZotDataSource"/>
        <set attribute="PoolName" value="ZotPool"/>
        <set attribute="Targets" value="${server-jmx-name}"/>
      </create>

    </weblogic8x.wlconfig>

Changing ALDSP End Points

I was trying to find a way to change ALDSP end points programmatically -- without having to go into the ALDSP console and manually retarget the physical stores.  I was hoping for a way to do this similar to the work I was doing with WebLogic JDBC pools and JMX.  Alas, there is no equivalent.  BEA did point out, however, the information is stored the XML configuration file:

admindomain/liquiddata/myaldspappLDConfig.xml

The file is read when the application is deployed.  So it is at least possible to make these changes, though it involves some XML hackery.

JMX Consoles: MC4J, EJAM, jManage

I used MC4J, a standalone JMX console that can access various server types -- including WebLogic which was my interest.  I also found, but did not try:

• an Eclipse plug-in called EJAM
• a web and command-line based tool called jManage

JUnit 4.0 BeforeClass Annotation

The JUnit @BeforeClass annotation has different semantics than the BeforeClass property in TestNG.  JUnit's version is invoked before any instances of the test case class are instantiated and, thus, can only access static features of the class.

Another gotcha to watch out for is that JUnit does not invoke any of the @BeforeClass or @Before methods of the superclasses of the test class.

I note that the Eclipse wizard for creating JUnit test cases does not permit you to specify a superclass -- is this a hint that the practice is discouraged?  Delegation appears to be the alternative, along with the use of static fields (ugh -- global variables).

Clay

I found a comfortable little Eclipse plug-in for editing SQL schemas:  Clay.

HTAs

I made a simple "hello world" HTA.