Understanding The Data From Your Serial Device

Before you can correctly configure the Software Wedge to parse and filter data received from your serial device, you must first have a complete understanding of the data that your device transmits. You should also be able to recognize what parts of the data are important to you and also completely understand what features of the Software Wedge can be used to manipulate your data. To use the Software Wedge effectively, you must think of each input from your device as a record of information containing one or more specific fields of data. The next step is to decide what features of the Wedge can be used to consistently separate each data record from the next and also what features can be used to separate or parse individual data fields within a record from the next field. Finally you need to identify which fields within each record are important to your application and which fields should be removed or ignored.

The best place to start is to refer to the users manual for the device that you are using. The users manual should have a section that describes the structure and contents of all data that can be outputted from the device. If you do not have a users manual for your device or if the output data structure is not described, you can use the Port-Analyze dialog box in the Software Wedge or the HyperTerminal program that is provided with Windows to manually analyze the output from your device by actually viewing the data that is transmitted from it.

When you configure the Software Wedge for a particular device you must define the input record structure for your serial data to the Wedge by selecting "Input Data Record Structure" from the Define menu. When defining the input record structure, you specify a "Start Of Record Event", an "End Of Record Event" and a general record structure for each data record (i.e. single field records, multiple delimited fields, or multiple fixed length fields).

With some devices, the record structure may be immediately obvious. For example, the following data record contains three numeric data fields delimited by commas and terminated by a carriage return-linefeed pair:

110,250,801<CrLf>

For this type of data record, you might select "Any Character Received" as the Start Of Record Event and "Carriage Return or CrLf Received" as the End Of Record Event and for the Record Structure you would choose "Multiple Delimited Data Fields" and specify that there are three data fields per record with a comma delimiter separating each field.

Note: The <CrLf> above refers to a carriage return and a linefeed character. A carriage return (Cr) will appear in the "Analyze" window in the Wedge as a musical note and a linefeed (Lf) will appear as a small rectangle with a circle inside it.

Other devices may output data with a record structure that is less obvious than the example above. Consider the following two possible (different) output records transmitted from one particular laboratory instrument:

Sample#1,*213**32****23**<CrLf>

Sample#2,*215**437**141**797**89**56<CrLf>

The two records appear similar in that they both are terminated by a carriage return-linefeed and they both seem to contain multiple delimited data fields. A minor problem with the above two records is that they both contain a different number of data fields. Another complication is that they seem to contain both a comma and one or more asterisks as delimiter characters. In the first record it appears that we have four data fields: "Sample#1", "213", "32" and "23"

In the second record it appears that we have seven data fields:

"Sample#2", "215", "437", "141", "797", "89" and "56"

There are actually several ways to configure the Wedge to correctly parse both records in the above example. The most elegant method is to use the "Pre-Input Character Translation Table" to modify the delimiters so that they are all the same. For example, we could translate all asterisks to commas to end up with the two records shown below:

Sample#1,,213,,32,,,,23,,<CrLf>

Sample#2,,215,,437,,141,,797,,89,,56<CrLf>

Now both records obviously have multiple, delimited, data fields with a carriage return-linefeed signaling the end of each record. We now determine the maximum number of fields in a record by adding up the number of delimiters in each record and adding one. (i.e. the first record has 11 fields and the second record has 13 fields therefore the maximum number of data fields is 13).

Now, when you define the record structure in the Software Wedge you would select "Any Character Received" as the Start Of Record Event and "Carriage Return or CrLf Received" as the End Of Record Event and for the Record Structure you would choose "Multiple Delimited Data Fields" and specify that the maximum number of data fields per record is 13 with a comma delimiter separating each field.

When parsing records with multiple delimited data fields, if you choose a space character as your delimiter, then consecutive spaces would be treated as a single delimiter. Therefore, in the above example, if instead of translating asterisks to commas, you were to translate both commas and asterisks to spaces, and then you chose the space character as your delimiter, you would effectively end up with the two records shown below with each field now separated by a single (space) delimiter:

Sample#1 213 32 23 <CrLf>

Sample#2 215 437 141 797 89 56<CrLf>

Suppose you had a device that normally sent its output to a serial printer in the following manner with more than one line of data:

Device# Height Width Length <CrLf>

--------------------------------------------------------- <CrLf>

1 23 12 24 <CrLf>

2 27 13 8 <CrLf>

3 27 13 9 <CrLf>

4 27 13 8 <CrLf>

--------------------------------------------------------- <CrLf>

<FF>

(<CrLf> represents a carriage return-linefeed and <FF> represents the form feed character)

You can still think of the entire output shown above as a single record even though it contains several lines of data or you can think of each individual line as a complete record. (In fact, you could even think of each word or number outputted from the device as a complete, single field, data record).

Again, there are hundreds of different ways to parse the entire output from the device using the different features available in the Software Wedge.

If you wanted to capture only the lines with numeric data and parse each of these lines into records with four fields, one way to accomplish this would be to specify the "Start Of Record Event" as "Numeric Character Received" and then specify the "End Of Record Event" as "Carriage Return or CrLf Received". Finally you would define the record structure as "Multiple Delimited Data Fields" with four fields per record and a space as the delimiter character. Because you chose "Numeric Character Received" as the "Start Of Record Event" and because there are no numeric characters in the first, second, seventh and eighth lines of data, these lines would be ignored.

If you wanted to capture the numeric data as above but with all 16 numbers in one record, you could again specify the "Start Of Record Event" as "Numeric Character Received" and then specify the "End Of Record Event" as "Special Character Received" and select the Dash ("-") character as the special character . Next, using the Pre-Input Character Translation Table, you would translate all carriage returns and linefeeds to spaces and finally you would define the record structure as "Multiple Delimited Data Fields" with 16 fields per record and the space character as the delimiter.

Once you understand all of the features provided in the Wedge, you can get very creative with how you deal with the data from your instruments.

The important points to remember are that the Wedge always waits until the selected "Start Of Record Event" occurs before it starts reading in any data after which it keeps reading in characters until the specified "End Of Record Event" occurs. When the "End Of Record Event" occurs, the Wedge takes the data record that it just received and parses it according to the definition of the Record Structure that you selected (i.e. Single Field, Multiple Delimited Fields or Multiple Fixed Length Fields). For each field that you have defined, the Wedge applies the chosen field Filter, Math Function and Format Expression. Finally the Wedge passes each field to the target application in sequence sending each field followed by the field's "Postamble" that you defined.

In many cases you could define the structure of your input data in many different ways. For example if your device transmitted the following data in bursts with some time between bursts, you could define the "End Of Record Event" as either "Carriage Return or CrLf Received" or as "Time Delay Between Records".

1, 23,12,24<CrLf>

2,27 ,13,8<CrLf>

3,27,13,9<CrLf>

If you specified "Carriage Return or CrLf Received" as the "End Of Record Event" then the Wedge would see the data as three records with four fields per record.

If you specified "Time Delay Between Records" and also used the "Pre-Input Translation Table" to translate carriage returns to commas and linefeeds to "Nul" or "Ignore", then the data would appear to the Wedge as follows:

1, 23,12,24,2,27,13,83,27,13,9

In other words you can think of the original data above as either three records containing four fields each or as a single record with 12 fields.

Again, as long as you understand the data from your serial device and have a clear understanding of the full capabilities of the Software Wedge, you can transform almost any serial data collection problem into an extremely simple task.