Its general structure is
Let us consider a recording of two EOG channels, one EEG channel and one EMG channel of 30 minutes sampled at 200 Hz. Each sample is coded as an integer of two bytes. We have decided that the record has a duration of 10 seconds. So, a record is composed by 2000 samples for each signal (8000 samples, i.e. 16000 bytes). Thirty minutes of recording will need 180 records. The file will have the following structure
The file will have 256 + (256 * 4) + 2880000 = 2881280 bytes.
Each signal can be sampled at a different rate but this rate has to be maintained. It will be indicated in the header of each signal. The detailed structure has been copied from EDF Technical specification
Figure 1. Detailed digital format of the header record (upper block, ASCII's only) and of each subsequent data record (lower block, integers only). Note that each one of the ns signals is characterised separately in the header. HEADER RECORD 8 ascii : version of this data format (0) 80 ascii : local patient identification 80 ascii : local recording identification 8 ascii : startdate of recording (dd.mm.yy) 8 ascii : starttime of recording (hh.mm.ss) 8 ascii : number of bytes in header record 44 ascii : reserved 8 ascii : number of data records (-1 if unknown) 8 ascii : duration of a data record, in seconds 4 ascii : number of signals (ns) in data record ns * 16 ascii : ns * label (e.g. EEG FpzCz or Body temp) ns * 80 ascii : ns * transducer type (e.g. AgAgCl electrode) ns * 8 ascii : ns * physical dimension (e.g. uV or degreeC) ns * 8 ascii : ns * physical minimum (e.g. -500 or 34) ns * 8 ascii : ns * physical maximum (e.g. 500 or 40) ns * 8 ascii : ns * digital minimum (e.g. -2048) ns * 8 ascii : ns * digital maximum (e.g. 2047) ns * 80 ascii : ns * prefiltering (e.g. HP:0.1Hz LP:75Hz) ns * 8 ascii : ns * nr of samples in each data record ns * 32 ascii : ns * reserved DATA RECORD nr of samples[1] * integer : first signal in the data record nr of samples[2] * integer : second signal .. .. nr of samples[ns] * integer : last signal
Our goal is to create some function in Scilab to read the header of EDF files to introduce the topic of loading files from external origin. The work will be developed on a file ( `osas.edf') that is included as an example by Bob Kemp, one of the authors of EDF, and that contains a segment of a polysomnographic recording.
First of all, we have to read the first global header to know the number of signals that forms the record.
clear()
name = xgetfile('*.*') // name of file
printf ("File selected : %s",name); // file selected
fid = mopen (name,'rb'); // opens file
mseek(0,fid,'end'); // goes to the end
totalbytes = mtell(fid); // reads bytes
mseek(0,fid) // goes beginning
header = mgetstr (256, fid); // reads header
version = part (header, 1:8); // version
patientid = part (header, 9:88); // patient ident.
recordid = part (header, 89:166); // record ident.
startdate = part (header, 169:176); // date
starttime = part (header, 177:184); // time
nbytesheader = eval (part (header, 185:192)); // bytes of header
reserved = part (header , 193:236); // reserved field
ndatarecords = eval ( part (header, 237:244)); // data records
duration = eval ( part (header, 245:252)); // duration
nsignals = eval ( part (header, 253:256)); // number of sign.
We extracted the information about duration of record in seconds as well as the number of signals. Numeric variables are converted with `eval'. We know the size of the file (totalbytes), the number of data records (ndatarecords) and the number of signals (nsignals), and we can calculate the number of bytes composing each record. Now we are going to read the header of the signals
for k=1:nsignals // labels
label(k)=mgetstr(16,fid);
end
for k=1:nsignals // transducers
transducer(k)=mgetstr(80,fid);
end
for k=1:nsignals // phys. dimensions
physdim(k)=mgetstr(8,fid);
end
for k=1:nsignals // physical minimum
physmin(k)=mgetstr(8,fid);
end
for k=1:nsignals // physical maximum
physmax(k)=mgetstr(8,fid);
end
for k=1:nsignals // digital minimum
digmin(k)=mgetstr(8,fid);
end
for k=1:nsignals // digital maximum
digmax(k)=mgetstr(8,fid);
end
for k=1:nsignals // prefiltering
prefilter(k)=mgetstr(80,fid);
end
for k=1:nsignals // samples of sign.
nsamples(k)=mgetstr(8,fid);
end
for k=1:nsignals // reserved
reserved(k)=mgetstr(32,fid);
end
physmin = evstr (physmin); // str >> value
physmax = evstr (physmax);
digmin = evstr (digmin);
digmax = evstr (digmax);
nsamples = evstr (nsamples);
We extract each value `nsignals' times, storing them as vectors. Afterwards, we convert them into integers when it is possible. We get a lot of information about each signal, that can be used to read data. The process of data extraction uses the command `matrix' that reshapes a vector or a matrix to a different size matrix. The figure represents the process in three data records, each one of them containing three signals.
We are going to include a code that reads data and stores them in a list of Scilab (we choose a list because each signal can be sampled at a different rate and consequently, the length of each trace can be different)
// End of each signal in record. Values stored in 'nend'
nend=[]
for k = 1:nsignals
dummy = sum(nsamples(1:k));
nend = [nend, dummy ];
end
// Beginning of each signal in record. Values stored in 'ninit'
ninit = [1,1+nend(1:$-1)];
// bytes in file except header
databytes = totalbytes - nbytesheader
// as ndatarecord can be -1 if unknown, we get it from data
bytesperrecord = nend($) * 2;
ndatarecords2 = databytes / bytesperrecord;
printf ("\ncontains %d signals (%d records of %d seconds) \n", ...
nsignals, ndatarecords2, duration);
str = sprintf ("Select a record to begin with 1-%d",...
ndatarecords2);
initrectoread = input (str)
str = sprintf ("Select a record to end with %d-%d",...
initrectoread,ndatarecords2)
endrectoread = input(str);
bytetostart = nbytesheader + (initrectoread -1) * bytesperrecord;
bytestoread = (endrectoread - initrectoread +1) * bytesperrecord;
integerstoread = bytestoread / 2;
mseek(int(bytetostart),fid);
data = mget (integerstoread,'s',fid);
data = matrix(data,nend($),length(data)/nend($));
signals = list();
for k= 1:nsignals
ndx = [ninit(k):nend(k)];
buffer = data(ndx,:);
buffer = matrix(buffer,1,size(buffer,1)*size(buffer,2));
signals ($+1) = buffer;
end
printf ("*******************************");
printf ("you got data in list <signals> ");
mclose('all')
This is the whole file that has been called `loadedf.sce'. It has not been well checked. Values are stored as they are in the file. Their real values can be deduced from the information contained in the header. The code is reproduced to maintain its parts assembled in such a way that you can copy and paste to a file that will be executed by Scilab. A copy of this file can be got here
clear()
name = xgetfile('*.*') // name of file
printf ("File selected : %s",name); // file selected
fid = mopen (name,'rb'); // opens file
mseek(0,fid,'end'); // goes to the end
totalbytes = mtell(fid); // reads bytes
mseek(0,fid) // goes to the beginning
header = mgetstr (256, fid); // read header
version = part (header, 1:8); // version
patientid = part (header, 9:88); // patient ident.
recordid = part (header, 89:166); // record ident.
startdate = part (header, 169:176); // date
starttime = part (header, 177:184); // time
nbytesheader = eval (part (header, 185:192)); // bytes of header
reserved = part (header , 193:236); // reserved field
ndatarecords = eval ( part (header, 237:244)); // data records
duration = eval ( part (header, 245:252)); // duration
nsignals = eval ( part (header, 253:256)); // number of sign.
for k=1:nsignals // labels
label(k)=mgetstr(16,fid);
end
for k=1:nsignals // transducers
transducer(k)=mgetstr(80,fid);
end
for k=1:nsignals // phys. dimensions
physdim(k)=mgetstr(8,fid);
end
for k=1:nsignals // physical minimum
physmin(k)=mgetstr(8,fid);
end
for k=1:nsignals // physical maximum
physmax(k)=mgetstr(8,fid);
end
for k=1:nsignals // digital minimum
digmin(k)=mgetstr(8,fid);
end
for k=1:nsignals // digital maximum
digmax(k)=mgetstr(8,fid);
end
for k=1:nsignals // prefiltering
prefilter(k)=mgetstr(80,fid);
end
for k=1:nsignals // samples of sign.
nsamples(k)=mgetstr(8,fid);
end
for k=1:nsignals // reserved
reserved(k)=mgetstr(32,fid);
end
physmin = evstr (physmin); // str >> value
physmax = evstr (physmax);
digmin = evstr (digmin);
digmax = evstr (digmax);
nsamples = evstr (nsamples);
// End of each signal in record. Values stored in 'nend'
nend=[]
for k = 1:nsignals
dummy = sum(nsamples(1:k));
nend = [nend, dummy ];
end
// Beginning of each signal in record. Values stored in 'ninit'
ninit = [1,1+nend(1:$-1)];
// bytes in file except header
databytes = totalbytes - nbytesheader
// as ndatarecord can be -1 if unknown, we get it from data
bytesperrecord = nend($) * 2;
ndatarecords2 = databytes / bytesperrecord;
printf ("\ncontains %d signals (%d records of %d seconds) \n", ...
nsignals, ndatarecords2, duration);
str = sprintf ("Select a record to begin with 1-%d",...
ndatarecords2);
initrectoread = input (str)
str = sprintf ("Select a record to end with %d-%d",...
initrectoread,ndatarecords2)
endrectoread = input(str);
bytetostart = nbytesheader + (initrectoread -1) * bytesperrecord;
bytestoread = (endrectoread - initrectoread +1) * bytesperrecord;
integerstoread = bytestoread / 2;
mseek(int(bytetostart),fid);
data = mget (integerstoread,'s',fid);
data = matrix(data,nend($),length(data)/nend($));
signals = list();
for k= 1:nsignals
ndx = [ninit(k):nend(k)];
buffer = data(ndx,:);
buffer = matrix(buffer,1,size(buffer,1)*size(buffer,2));
signals ($+1) = buffer;
end
printf ("*******************************");
printf ("you got data in list <signals> ");
mclose('all')
Now we can check the code with the file included in Bob Kemp's page. You have to discharge 2.5 Mb of data and unzip the file.
-->exec('loadedf.sce',-1)
File selected : /home/j/jj_dat/edffiles/osas.edf
contains 17 signals (120 records of 10 seconds)
Select a record to begin with 1-120-->51
Select a record to end with 51-120-->55
*******************************
you got data in list <signals>
-->plot(signals(12)(1:400))
And the result is shown in figure 6.
If you select records from 1 to 120, you have to increase the memory dedicated to data with the command `stacksize'
-->stacksize(20000000)
-->clear
-->exec('edf.sce',-1)
File selected : /home/j/webpages/jj_dat/edffiles/osas.edf
contains 17 signals (120 records of 10 seconds)
Select a record to begin with 1-120-->1
Select a record to end with 1-120-->120
*******************************
you got data in list <signals>
-->save signals
We saved a huge file of 30 Mb that can be directly read with the next code. In the next part we are going to work with data contained in this file. It has the advantage of containing a wide range of signals. It is recommended that you store in your hard disk a copy of a file named `signals' that includes the file `osas.edf' in Scilab format. Of course although this format is very inefficient to store the huge amount of data included in `osas.edf', it can be easily read by Scilab. If you do not have enough memory, you can reduce the number of records (for example by choosing `1' and `10' as the records to begin and end). If you save your data in `signals' you will be able to run most procedures in the next chapters. The commands to recover `signals' are
-->stacksize(20000000) -->load signals
Now imagine that we begin a new session and we want to plot the data contained in the file. We can plot data in a way similar to a polysomnographic recording
stacksize(20000000); // increases storage area load signals; // signals had been previously generated for k=1:16; // adjusts subscreen for each trace xsetech([0,(k-1)/16,1,1/16]); // plots each signal without axis plot2d(1:20000,signals(k)(1:20000),1,'020'); end;
The labels that indicate where the signals come from can be recovered from 'label'
-->label label = !Fp1-M2 ! ! ! !C3-M2 ! ! ! !O1-M2 ! ! ! !Fp2-M1 ! ! ! !C4-M1 ! ! ! !O2-M1 ! ! ! !M2-M1 ! ! ! !Pos8-M1 ! ! ! !Pos18-M1 ! ! ! !EMGsubmental ! ! ! !EMGlinked ! ! ! !ECG ! ! ! !Airflow ! ! ! !Chest ! ! ! !Abdomen ! ! ! !SaO2 ! ! ! !Hypnogram !
In this chapter we have been able to read an external file, to save and recover data in Scilab format, and to plot them in a way similar to a polygraphic recording. The traces read in this chapter (stored in the file `signals') are going to be used in the second part of the manual as a benchmark where we will check the different procedures of signal treatment.
