Parallel jobs on Windows fail with APT_IOPort::readBlkVirt;error

APT_IOPort::readBlkVirt Error Screenshot
APT_IOPort::readBlkVirt Error Screenshot

This a known error for windows systems and applies to DataStage and DataQuality jobs using the any or all the three join type stages (Join, Merge, and Lookup).

Error Message

  • <<Link name>>,0: APT_IOPort::readBlkVirt: read for block header, partition 0, [fd 4], returned -1 with errno 10,054 (Unknown error)

Message ID

  • IIS-DSEE-TFIO-00223

Applies To

  • Windows systems only
  • Parallel Engine Jobs the three join type stages (Join, Merge, and Lookup). It does not apply to Server Engine jobs.
  • Infosphere Information Server (IIS), Datastage and DataQuality 9.1 and higher

The Fix

  • Add the APT_NO_IOCOMM_OPTIMIZATION in project administrator and set to blank or 0. I left it blank so it would not impact other jobs
  • Add the environment variable to the job producing the error and set to 1

What it APT_NO_IOCOMM_OPTIMIZATION Does

  • Sets the use of shared memory as the transport type, rather than using the default sockets transport type.
  • Note that in most cases sockets transport type is faster, so, you likely will not to set this across the project as the default for all job. It is best to apply it as necessary for problematic jobs.

Related References

InfoSphere DataStage and QualityStage, Version 9.1 Job Compatibility

IBM Support, JR54078: PARALLEL JOBS ON WINDOWS FAIL WITH APT_IOPORT::READBLKVIRT; ERROR

IBM Support, Information Server DataStage job fails with unknown error 10,054.

 

Aginity For Netezza – How to Generate DDL

Aginity, Aginity for Netezza, Netezza, PureData, DDL, SQL
Aginity

How to Generate Netezza Object DDL

In ‘Aginity for Netezza’ this process is easy, if you have a user with sufficient permissions.

The basic process is:

  • In the object browser, navigate to the Database
  • select the Object (e.g. table, view, stored procedure)
  • Right Click, Select ‘Script’ > ‘DDL to query window’
  • The Object DDL will appear in the query window
Create DDL to Query Window
Create DDL to Query Window

Related References

 

[HY000] ERROR: Base table/view has changed (datatype); rebuild view

Error
Error

Error Message

ERROR [HY000] ERROR: Base table/view ‘BLOG_LIST_TBL’ attr ‘BLOG_ID’ has changed (datatype); rebuild view ‘BLOG_LIST_VW’

What does the Error Mean?

This error means that an underlying table used by the view has changed in a way, which made the view invalid, as a result, the view must be rebuilt to reflect the new table definition.

How to Rebuild the view

In ‘Aginity for Netezza’ this process is easy, if you have a user with sufficient permissions.  The basic process is:

  • Navigate to and select the view to be rebuilt
  • Right Click, Select ‘Script’ > ‘DDL to query window’
Create DDL to Query Window
Create DDL to Query Window
  • Once the ‘Create or Replace View’ SQL has generated, click within the SQL statement (without highlighting)
  • Press Ctrl+F5 or navigate to Execute > ‘Execute as a Single batch
Execute SQL as a Single Batch
Execute SQL as a Single Batch
  • Verify the ‘Create or Replace View’ SQL executes successfully
  • Then, run a simple select against the view and that Select runs without producing the HYOOO Base Table/View Error

DataStage – IIS-DSEE-TBLD-00008- Processing Input Record APT_Decimal Error

IIS-DSEE-TBLD-00008 apt decimal error before disabling combination
IIS-DSEE-TBLD-00008 apt decimal error

This another one of those nebulas error messages, which can cost a lot of time in research, if you don’t know how to simplify the process a bit.  However, determining where the error is can be a bit of a challenge if you have not encountered this error before and figured out the trick, which isn’t exactly intuitive.

In this case, as it turned out after I had determined where the error was, it was as simple as having missed resetting the stage variable properties, when the other decimal fields increased.

How to identify where this error occurs?

Disabling the APT_DISABLE_COMBINATION environment variable by:

  • adding the APT_DISABLE_COMBINATION environment variable to the job properties
  • setting the  APT_DISABLE_COMBINATION environment variable it to true in the job properties
  • compiling the job and running the job again

This approach will, usually, provide a more meaningful identification of the stage with the error.

Note:  Please remember to remove the APT_DISABLE_COMBINATION environment variable before moving it to testing and/or releasing your code in production.

Message ID

IIS-DSEE-TBLD-00008

Error Message with combine enabled:

APT_CombinedOperatorController(1),0: Exception caught in processingInputRecord() for input “0”: APT_Decimal::ErrorBase: From: the source decimal has even precision, but non-zero in the leading nybble, or is too large for the destination decimal… Record dropped. Create a reject link to save the record if needed.

Error message with combine disabled

Tfm_Measures_calc,0: Exception caught in processingInputRecord() for input “0”: APT_Decimal::ErrorBase: From: the source decimal has even precision, but non-zero in the leading nybble or is too large for the destination decimal… Record dropped. Create a reject link to save the record if needed.

IIS-DSEE-TBLD-00008 apt decimal error after disabling combination
IIS-DSEE-TBLD-00008 apt decimal error after disabling combination

Note: Measures_calc is the stage name

Related References

Infosphere – decimal_from_string Conversion Error

IBM Infosphere - decimal_from_string Conversion Error
decimal_from_string Conversion Error

This is another one of those nebulas error, which can kick out of DataStage, DataQuality, and/or DataClick.  This error can be particularly annoying because it doesn’t identify the field or even the precise command, which is causing the error.  So, there can be more than a field and/or more than one command causing the problem.

Error

Conversion error calling conversion routine decimal_from_string data may have been lost

Resolution

To resolve this error, check for the correct formatting (date format, decimal, and null value handling) before passing to datastage StringToDate, DateToString, DecimalToString or StringToDecimal functions.  Additionally, even if the formatting is correct, you may need to embed commands to completely clear the issue.

Example

Here is a recent example of command embedding, which has clear the issue, but I’m sure you will need to this concept in other ways to meet all your needs.

DecimalToString( DecimalToDecimal( <>,’trunc_zero’),”suppress_zero”)

How to suppress a Change_Capture_Host warning in Datastage

Change Capture Host Warning (IIS-DSEE-TFXR-00017)
Change Capture Host Warning (IIS-DSEE-TFXR-00017)

Occasionally, I run into this Change Capture Host defaulting warming, so, I thought this information may be useful.

Event Type

  • Warning

Message ID

  • IIS-DSEE-TFXR-00017

Example Message

  • Change_Capture_Host: When checking operator: Defaulting “<<FieldName>>” in transfer from “beforeRec” to “outputRec”.

Setting Variable

  • Set APT_DISABLE_TFXR0017=1
  • This environment variable can be added either at the project level or at the job level.

Alternative Solution

  • Within the Change Capture stage properties:
    • Stage tab
    • Option
    • Property: “Change Mode
    • Value:  “Explicit key, All Values”.

What does “ERROR: pg_atoi: error in : can’t parse ” mean?

This is one of those vague messages, which takes a little time to run down, but once you understand it, is easily fixed.  Basically, this error happens because you have mismatch and incompatible fields.

Question

What does “ERROR: pg_atoi: error in <ascii>: can’t parse <ascii>” mean?

Answer

The error is the result of an implicit conversion being performed on a character field (varchar/char) while being compared to a numeric field or value.

Vendor Reference Link