02 November 2013

DataStage Patterns–Generate Sequence of Numbers in a Parallel Transformer

A common requirement in the DataStage jobs is to create a sequence of unique numbers (row id in other words) in a parallel transformer stage. The obvious solution bellow does not work if the job is running on multiple partitions:

image

To overcome this the @PARTITIONNUM, @NUMPARTITIONS and @INROWNUM system variables should be used in a transformer derivation like this:

@PARTITIONNUM + (@NUMPARTITIONS * (@INROWNUM -1))

image

No comments: