ASCII files

Often, we have to recover data data that have been generated by other applications. Perhaps the easiest way to share data with other devices, unless they have a compatible format of file available is exporting data in ASCII format. Some neurophysiological equipment create ASCII files as an option. Usually, values are stored as real values with a structure similar to a matrix where each column represents a channel. To get data from an ASCII file we are going to use a file named `test.dat' in the directory where Scilab has access (it has been generated with Scilab but it can be created with any other program, such as a spreadsheet, storing data in text format, etc.). We can see its content from the shell prompt

[j@localhost test]$ cat test.dat
 0.2113249 0.5608486 0.3076091 0.5015342 0.2806498
 0.7560439 0.6623569 0.9329616 0.4368588 0.1280058
 0.0002211 0.7263507 0.2146008 0.2693125 0.7783129
 0.3303271 0.1985144 0.3126420 0.6325745 0.2119030
 0.6653811 0.5442573 0.3616361 0.4051954 0.1121355
 0.6283918 0.2320748 0.2922267 0.9184708 0.6856896
 0.8497452 0.2312237 0.5664249 0.0437334 0.1531217
 0.6857310 0.2164633 0.4826472 0.4818509 0.6970851
 0.8782165 0.8833888 0.3321719 0.2639556 0.8415518
 0.0683740 0.6525135 0.5935095 0.4148104 0.4062025

The next code of Scilab can recover data

-->fid = file ('open','test.dat','unknown')         
 fid  =
 
    2.  
 
-->data = read (fid, -1, 5)
 data  =
 
!   0.2113249    0.5608486    0.3076091    0.5015342    0.2806498 !
!   0.7560439    0.6623569    0.9329616    0.4368588    0.1280058 !
!   0.0002211    0.7263507    0.2146008    0.2693125    0.7783129 !
!   0.3303271    0.1985144    0.312642     0.6325745    0.211903  !
!   0.6653811    0.5442573    0.3616361    0.4051954    0.1121355 !
!   0.6283918    0.2320748    0.2922267    0.9184708    0.6856896 !
!   0.8497452    0.2312237    0.5664249    0.0437334    0.1531217 !
!   0.685731     0.2164633    0.4826472    0.4818509    0.6970851 !
!   0.8782165    0.8833888    0.3321719    0.2639556    0.8415518 !
!   0.068374     0.6525135    0.5935095    0.4148104    0.4062025 !
 
-->file ('close' , fid)

Some aspects of this code deserve a further comment: `file' is used to open and close the file and values are recovered by `read'. Each raw is called a register. As we know that each register has five values, the third option indicates that each register is composed by five values. Since we do not know the number of registers we adjust the second parameter to `-1' and the file is read until the end of its content.

This code works well when a file is made up by values. Frequently, files contain some text indicating things like patient data, filters, sampling rate or other similar information. When you process these files, the above code produces some difficulties that can be avoided in several ways.

1. Code can be improved by selectively reading some lines. Even we can read the information (filters, patient...) contained in the file.
2. The file can be edited eliminating by hand the lines at which you are not interested.
3. Finally, Scilab includes the command `fscanfMat('filename')' to read `filename' in such a way that first non-numeric lines of the file are ignored. You can also print matrices in ASCII format with `fprintfMat'. Both functions allow an exchange of information with many other programs or devices. Notice that `fscanfMat' only ignores the first non numeric lines. If non-numeric lines are mixed together with numeric values, this command will only read until it reaches a non-numeric line.

We modify slightly `test.dat' by including some lines of text with any editor and from the shell prompt we type

[j@localhost test]$ cat test2.dat
This is the first line
and this is the second line
 0.2113249 0.5608486 0.3076091 0.5015342 0.2806498
 0.7560439 0.6623569 0.9329616 0.4368588 0.1280058
 0.0002211 0.7263507 0.2146008 0.2693125 0.7783129
 0.3303271 0.1985144 0.3126420 0.6325745 0.2119030
 0.6653811 0.5442573 0.3616361 0.4051954 0.1121355
after five numeric lines...
 0.6283918 0.2320748 0.2922267 0.9184708 0.6856896
 0.8497452 0.2312237 0.5664249 0.0437334 0.1531217
 0.6857310 0.2164633 0.4826472 0.4818509 0.6970851
 0.8782165 0.8833888 0.3321719 0.2639556 0.8415518
 0.0683740 0.6525135 0.5935095 0.4148104 0.4062025

And from Scilab

-->data2 = fscanfMat ('test2.dat')
 data2  =
 
!   0.2113249    0.5608486    0.3076091    0.5015342    0.2806498 !
!   0.7560439    0.6623569    0.9329616    0.4368588    0.1280058 !
!   0.0002211    0.7263507    0.2146008    0.2693125    0.7783129 !
!   0.3303271    0.1985144    0.312642     0.6325745    0.211903  !
!   0.6653811    0.5442573    0.3616361    0.4051954    0.1121355 !

Notice that this function eliminates lines properly at the beginning but the reading ceases when the line beginning with `after five...' is reached.

To export data with a simple format, we can use the command `fprintfMat'

-->data = rand (20,5,'normal')
 data  =
 
! - 0.7616491  - 0.7004486    0.0116391    0.5163254    0.2685505 !
!   1.4739762  - 0.8262233  - 1.4344474    1.0422456    0.8780630 !
!   0.8529775    1.1323911    1.748736     2.4976094  - 1.0179838 !
!   0.7223316  - 0.2330131    0.1645912    0.6450695  - 1.9598192 !
!   0.6380837  - 0.2343923    0.9182207  - 0.4483985    0.0927515 !
!   0.2546697    1.4027611    0.0259119  - 0.7218988    1.2311603 !
! - 0.6834217    0.3268713    0.7953703  - 0.5485232    0.6697461 !
!   0.8145126    0.0613533  - 1.3770621    1.1316247    0.2823040 !
! - 0.1884803  - 0.1890526  - 0.9063738  - 1.3667445  - 2.0049863 !
! - 1.0327357    0.4249849    1.2296215  - 1.3850463    1.1758127 !
! - 0.9239258  - 0.7460990  - 0.4577385  - 1.6280467  - 0.2570480 !
!   2.7266682  - 1.721103   - 0.5875092    0.7757721  - 0.1595813 !
! - 1.7086774  - 1.7157583    0.2301981  - 1.1049895    0.7131577 !
!   0.0698768    0.1023021  - 0.2563031    0.7604477  - 1.2140245 !
! - 1.3772844  - 1.2858605    1.1937458  - 0.6588100  - 2.343241  !
! - 0.1728369    0.6107784    1.8655072    0.1068464  - 2.0499752 !
! - 0.6019869  - 0.3778182  - 1.3189198    0.7469882  - 0.4183469 !
! - 1.5619521    2.5749104  - 0.8575199    0.0979236  - 0.2768419 !
! - 0.3888655  - 0.4575284    0.2973099    0.2946056  - 1.2710279 !
!   0.6543045  - 0.6453261  - 1.5404673    1.1474321  - 0.9639153 !
 
-->fprintfMat('test3.dat',data,'%15.10f')

And from the shell prompt

 

[j@localhost test]$ cat test3.dat
  -0.7616490874   -0.7004486348    0.0116391089    0.5163254498    0.2685505359
   1.4739762439   -0.8262233278   -1.4344473643    1.0422456176    0.8780630158
   0.8529775253    1.1323910660    1.7487360025    2.4976094313   -1.0179837547
   0.7223315777   -0.2330131163    0.1645911738    0.6450695409   -1.9598191507
   0.6380837232   -0.2343923439    0.9182207350   -0.4483984815    0.0927514850
   0.2546696793    1.4027611492    0.0259118507   -0.7218988418    1.2311603033
  -0.6834217347    0.3268713158    0.7953703383   -0.5485232377    0.6697460667
   0.8145126073    0.0613532795   -1.3770621196    1.1316247373    0.2823039722
  -0.1884802915   -0.1890525732   -0.9063738301   -1.3667444990   -2.0049863130
  -1.0327357336    0.4249848564    1.2296215292   -1.3850462843    1.1758126732
  -0.9239258062   -0.7460990484   -0.4577385334   -1.6280466584   -0.2570479823
   2.7266682098   -1.7211029770   -0.5875091875    0.7757720724   -0.1595812688
  -1.7086773593   -1.7157583243    0.2301980611   -1.1049895223    0.7131576895
   0.0698768244    0.1023020996   -0.2563031358    0.7604476679   -1.2140244569
  -1.3772844122   -1.2858605195    1.1937458319   -0.6588100458   -2.3432410362
  -0.1728369461    0.6107784158    1.8655071775    0.1068464105   -2.0499751583
  -0.6019868859   -0.3778182298   -1.3189197768    0.7469881736   -0.4183469030
  -1.5619521175    2.5749103741   -0.8575198721    0.0979236367   -0.2768419233
  -0.3888655235   -0.4575283716    0.2973098569    0.2946055949   -1.2710279057
   0.6543044610   -0.6453261045   -1.5404673140    1.1474320785   -0.9639152877

Notice that the resolution is determined by the string that defines format in `fprintfMat'. This format can be easily read by many programs.