Handling floating point numbers

The internal expression processor operates on internal floating point fields in their native, hexadecimal floating point format. External floating point fields are converted to double-precision internal format before being processed as if they were internal floating point fields. If two fields in a comparison have different precisions, then the field with the greater precision is rounded to the lesser precision, and the comparison is performed using the lesser precision. Because internal floating point fields are converted to external (decimal) format and passed to REXX as strings, and external floating point fields are also passed as strings, the result of evaluating a comparison can differ between REXX and the internal expression processor.

To ensure consistency, pay attention to the following points when coding expressions involving floating point fields: