TDI Tutorial

TDI is the expression language for MDSplus. It is used to manipulate data within the Traverser, the Scope and anywhere data is accessed. It is used to access, process and store the data. It is sometimes referred to as an expression evaluator.

This tutorial is intended to provide familiarity with the syntax and some of the functions of TDI, and point to references for further information. The more advanced topics include various ways of building signals and calling routines developed in other languages from TDI.

(this search is not yet working):

Simple Interactive Examples
Working with Arrays in TDI
Working with Signals in TDI
Example of TDI in a Scope
Writing a simple TDI Function (a .FUN)
Writing a FORTRAN program to be called from TDI
Documentation on user-written .FUN's
Efficiency
Slicing Arrays
CAVEATS
TDI Reference Pages

Simple Interactive Examples

(The following assumes you have done a "module load nstx" first)

TDI can be executed from the command line by:


tdic
TDI> treeopen("pspec_pc",126725)
265388041
TDI> _inputsig = \Xeus_image ;    
TDI> _r = make_xeus_spectra( _inputsig);

(A minimum number of examples will be shown here. You should try several of your own, including some with errors, to become familiar with error messages.)

One way to try TDI commands interactively is from IDL. Using the TDIC program, distributed with MDSplus, is probably easier.

First, as with any MDSplus access from IDL, you must be connected to an MDSplus server, and have a tree open.

(NOTE that you must be authorized to connect to most MDSplus servers).

The users entries are in bold type:

 
    $ idl 
    IDL> MDSCONNECT, 'europa.pppl.gov:8501'	; unnecessary if on the server 
    IDL> MDSOPEN, 'nstx', 104500		; tree and shot (or ID) number 
    IDL> tm1 = MDSVALUE( "\TMINUS1" )		; \TMINUS1 is a tag in this tree 
    IDL> help, tm1
      TM1             FLOAT     =      -1.00000

In the following examples, the TDI expression is the quoted argument to MDSVALUE.

TDI supports numerical expressions:

   IDL> print, MDSVALUE( "\TMINUS1 * 10 - 2" )
     -12.0000

What do you think you'll get from the following?

   IDL> print, MDSVALUE( "\TMINUS1 - 2 * 10" )

(The rules for precedence in TDI are well defined).

TDI supports a variety of functions (like many of those in FORTRAN-90 and C):

   IDL> print, MDSVALUE( "(ABS(\TMINUS1)+1)^2" )
      4.00000

You may perform these operations on tags that return arrays, as well (\ip is an MDSplus tag in the WF tree for NSTX):

   IDL> plot,MDSVALUE('\wf::ip/1000')

For efficiency, or code clarity, you may wish to store results from TDI expressions into TDI variables (which must begin with an underscore), e.g.,

   IDL> dummy = MDSVALUE('_ip=\wf::ip/1000')
   IDL> print,MDSVALUE('maxval(_ip)')
	 1.16095
   IDL> print,MDSVALUE('maxval(cos(_ip))')
	 1.00000

Yes, there is integer arithmetic in TDI:

   IDL> print,MDSVALUE("_val=7/4")
           1
   IDL> print,MDSVALUE("_val=7/float(4)")
	 1.75000

There are many constant intrinsics in TDI:

   IDL> print,MDSVALUE("$PI")
      3.14159
   IDL> print,MDSVALUE("$SHOT-1")
      104499

GETNCI is useful for finding node characteristics.

Find the full path from the top of the tree for nodes containing IP:

   IDL> t=MDSVALUE('_t=getnci("...*IP*","fullpath","ANY")')

Using a TDI function called FINDTAGS, you can find all the nodes that have "IP" in the tag:

   IDL> nodesFound = MDSVALUE( '_n=findtags($)','*IP*' )

Working with Arrays in TDI

Arrays can be defined explicitly:

   IDL> print,MDSVALUE("_a=[6,5,4,3,2,1]")
           6           5           4           3           2           1
   IDL> print,MDSVALUE("_a=(6:1:-1)")	; alternate method
           6           5           4           3           2           1

Arrays can be used as arguments in functions or expressions. Array can be subscripted. Note that array indices are zero-based.

   IDL> print,MDSVALUE("minloc(_a)")	; index of minimum value in _a
           5
   IDL> print,MDSVALUE("_a[ minloc(_a) ]")
           1
   IDL> print,MDSVALUE("ubound(_a)")	; highest index in _a
           5
   IDL> print,MDSVALUE("mean(_a*10)")
          35
   IDL> print,MDSVALUE("_a[5]") 
           1
   IDL> print,MDSVALUE("mean(_a[0:2])")
           5

Note that you can NOT use an array subscript on the left-hand side of an equal sign in TDI:

   IDL> print,MDSVALUE("_a[5]=3")
   % MDSVALUE: %TDI$-E-INV_OPC, Invalid operator code in a function
     %TDI Error in EQUALS(_a[5], 3)
     %TDI Error in EVALUATE(_a[5] = 3)
     %TDI Error in EXECUTE("_a[5]=3")

The dimension for a variable might be defined with the BUILD_RANGE function:

   IDL> print,MDSVALUE("BUILD_RANGE(1,17,4)")
           1           5           9          13          17
   IDL> print,MDSVALUE("(1:17:4)")           ; short-hand method
           1           5           9          13          17

Similar notation can be to skip data before it is returned to your program, e.g., to return every 10th data point of a MDSplus signal:

   IDL> MDSOPEN, 'wf', 109070
   IDL> sparseData = MDSVALUE( "DATA(\ip)[0:*:10]" )

Since a signal involves a complicated descriptor, the DATA( ) function must be used. Here is a way to get the 10th frame from a 3-D signal of camera data:

   IDL> MDSOPEN, 'particles', 113458
   IDL> frame = MDSVALUE( "DATA(\sflip_3d)[*,*,10]" )

For non-signals, such as a dimension, DATA( ) is not necessary:

   IDL> sparseTime = MDSVALUE("DIM_OF(\ip)[0:*:10]")

Working with Signals in TDI

TDI supports many data types. The Signal data type is powerful and has several different looks. A Signal typically consists of an expression based on the raw data, the raw data itself, and expressions for each dimension.

For example, the TDI for a signal composed of a few digitizer counts (made artificially small here for simplicity) on a 1 KHz clock might look like:

   _sig = BUILD_SIGNAL( 5./4095 * $VALUE,   /* expression using RAW */
	                [5,2,1,4,2],        /* Raw data */
			[0,1,2,3,4]/1000.); /* Dimension */

An equivalent, but more efficient, way to store a time base (when there are realistic numbers of points) is with the BUILD_RANGE function (or similar TDI Intrinsic functions) that just store starting and ending points, and the delta X. E.g.:

   _sig = BUILD_SIGNAL( 5./4095 * $VALUE,     /* expression using RAW */
	                [5,2,1,4,2],          /* Raw data */
	     BUILD_RANGE(0.0, 0.004, 0.001)); /* Dimension */

(Note that $VALUE is a TDI Constant Intrinsic used in the data field of a signal to mean the raw field.)

If you don't wish to bother with the Raw field, you can build this signal as follows:

   _sig = BUILD_SIGNAL( 5./4095*[5,2,1,4,2],  
	                *,                    /* No Raw data */
	     BUILD_RANGE(0.0, 0.004, 0.001)); /* Dimension */

To explore signal building in IDL, try the following:

   IDL> s = MDSVALUE("_s=BUILD_SIGNAL( [5,2,1,4,2],"+ $
	             "*, BUILD_RANGE(0.0, 0.004, 0.001))" )       

   IDL> print,s
           5           2           1           4           2

   IDL> print, MDSVALUE("DIM_OF(_s)")
      0.00000   0.00100000   0.00200000   0.00300000   0.00400000

But you should build your signal with units, so :

   IDL> str='BUILD_SIGNAL( BUILD_WITH_UNITS([5,2,1,4,2],"Volts"),'          + $  ;DATA 
			 ','                                                + $  ;RAW
			 'BUILD_DIM( BUILD_WINDOW(0,4,0.0),'                + $  ;DIMENSION
				    'BUILD_RANGE(*, *, BUILD_WITH_UNITS(0.001,"Sec") )' + $  ;
				  ')'                                       + $  ;
			')' 		    
		    
   IDL> s = MDSVALUE("_s="+str)
   
   IDL> print,s
           5           2           1           4           2

   IDL> print, MDSVALUE("DIM_OF(_s)")
      0.00000   0.00100000   0.00200000   0.00300000   0.00400000

   IDL> print, MDSVALUE("UNITS_OF(_s)")
   Volts

   IDL> print, MDSVALUE("UNITS_OF(DIM_OF(_s))")
   Sec

A two-dimensional Signal could be definced as follows (assuming _image, _x, & _y are defined):

   IDL> str = "BUILD_SIGNAL( BUILD_WITH_UNITS(_image,'Photons'),," + $
	      "BUILD_WITH_UNITS(_x,'Sec'), BUILD_WITH_UNITS(_y,'cm') )", $

   IDL> s = MDSVALUE(str)

A $ can be used as a place holder in TDI expressions. The following shows a way of writing an MDSplus signal into an opened tree from IDL:

   IDL> MDSPUT, sigName,  $
          "BUILD_SIGNAL( BUILD_WITH_UNITS($,$),," + $
	                "BUILD_WITH_UNITS($,$), BUILD_WITH_UNITS($,$) )", $
           data, sigUnits, xAxis, xUnits, yAxis, yUnits

By default, the $ place holders are filled from left to right by the variables in the argument list.

When data types are mixed there are rules for the type of the result. E.g.,

   IDL> dum = MDSVALUE('_f=[.1,.2,.3]')
   IDL> dum = MDSVALUE('_c=CMPLX(2,3)')
   IDL> prod = MDSVALUE('_f*_c')
   IDL> help,prod
   PROD            COMPLEX   = Array[3]
   IDL> print,prod
   (     0.200000,     0.300000)(     0.400000,     0.600000)
   (     0.600000,     0.900000)

Example of TDI in a Scope

The figure above is the setup screen for a scope signal (brought up by holding down the right mouse button over a scope plot, and selecting "Setup Data Source...").

TDI is used in the fields labeled "Y Axis:", "X Axis:", and "Title:". The Y-axis is the average of two signals, represented by tags \ip1 & \ip2. The X-axis just uses the dimensions of one of the signals. The Title includes the units (of one of the signals) and appends the shot number by using the constant intrinsic for the shot number, $SHOT. Note that double slashes (//) are used for concatenating strings in TDI.

Writing a simple TDI Function (a .FUN)

In a text editor, create the following and save it in a file named testtag.fun:
NOTE that each line of TDI code is terminated with a semi-colon.

   FUN PUBLIC testtag( IN _c ) {

	_tm1 = \TMINUS1;
	_result = _tm1 * _c;

	write(*,'_c=',_c);
	write(*,'_tm1=',_tm1);
	write(*,'_result=',_result);

	RETURN (_result); 
   }

To use a new locally-written TDI function (.FUN file) in an MDSplus node or expression, it is necessary to put the file into the search-list. On VMS, this is defined by the logical name MDS$PATH; on Unix it is the environmental variable MDS_PATH. On VMS at PPPL, for example, the directory MDSplus$:[PPPL] is for user-written TDI of general interest. On VMS, the logical name MDS$USER_PATH is searched before MDS$PATH, so directories pointed to by it should be used for testing, or for routines that will only be used by the author.

Execute your TDI from IDL:

    $ idl
    IDL> MDSCONNECT, 'europa.pppl.gov:8501'	; unnecessary if on the server 
    IDL> MDSOPEN, 'nstx', 104500		; tree and shot (or ID) number 
    IDL> a=MDSVALUE("testtag(5)")
    _c=           5
    _tm1=       -1.0000E0
    _result=       -5.0000E0

Writing a FORTRAN program to be called from TDI

You may wish to add routines written in other languages, such as FORTRAN or C, to a shared library so they can be called from TDI (or IDL). Sometimes this is done for speed, or so Scope can perform some specialized function. This section shows how to link in a routine written in FORTRAN.

You may wish to Test your Fortran first.

Before you can call such a routine from TDI or IDL, you must put it in a shared image on VMS, a Shared Object module on Unix, or a DLL on Windows.

Then, create a logical name or environmental variable to point to the shared library:

On VMS:

 	$ DEFINE myshrdlib mydisk:[mydir]myshrdlib.exe

On Unix:

        % setenv myshrdlib mydisk/mydir/myshrdlib.so

In TDI, the way to call such a routine, named, say, integ, is:

  	_sts = myshrdlib->integ( _nels, REF(_x),  REF(_y), REF(_rInteg) );

In IDL:

        rInteg = FLTARR( n_elements(x) )
   	result = CALL_EXTERNAL( 'myshrdlib', 'integ',  $
                                 LONG(n_elements(x)), x, y, rInteg)

Documentation on user-written .FUN's

If conventions are followed for a documentation header in a TDI routine, as well as being easier for users to find information, web pages can be automatically generated to document systems, just as with IDL routines.

For example:

/*      SECONDS_SINCE.FUN
; PURPOSE:      Return a floating point number of seconds
;               given an input quadword time.
; CATEGORY:     Data Conversion
; CALLING SEQUENCE: seconds = SECONDS_SINCE(quadword-time [,_since-time-string])
; INPUTS:
;               quadword-time - a VMS time or array of times.
;
; OPTIONAL INPUT PARAMETERS: --
;               _since-time-string   - date to start from
;                                      default '1-JAN-1992'
; OUTPUTS:      --
; OPTIONAL OUTPUT PARAMETERS: --
; COMMON BLOCKS: --
; SIDE EFFECTS: --
; RESTRICTIONS: --
; PROCEDURE:    --
; MODIFICATION HISTORY:
;       JAS 14-OCT-1992 Initial coding.
*/
FUN     PUBLIC SECONDS_SINCE(IN _time, OPTIONAL IN _since) {
  _DELTA = 0Q;
  _ANS = 0.0;
  if (not PRESENT(_since)) _since = '1-JAN-1992';
  _stat = LIBRTL->LIB$CONVERT_DATE_STRING( DESCR(_since),REF(_DELTA));
  _q_times = _time - _delta;
  _q_times = _q_times / 100000Q; /* hundredths of a second */
  _ans = FLOAT(_q_times)/(100.);
  return(_ans);
}

(Links to user-written TDI to-be-provided.)

Efficiency

Where your TDI is executed can affect the speed of your call.

If, for example, you are getting a digitizer signal, and the TDI is not available on your remote host, the 16-bit values will be converted to floating point and sent as 32-bit numbers (or 64-bit for double precision), as will the time signal (DIM_OF). If the TDI is executed on your remote computer, rather than on the MDSplus host, just 2 bytes per data point will be transfered (rather than 4, or 8 if double-precision), and t0, delta t and number of points will be transfered, rather than the entire time array. So, approximately 1/4 the amount of data is transfered in this example if the conversion to floating point is done on your remote computer.

On the other hand, if the TDI expression includes a dozen different signals, executing the TDI code on the MDSplus server will result in substantially less data being transfered.

Network transfer speeds are typically the slowest part of data access, but your conditions may be different. The relative speed and load of the client and server computers may also be a consideration in deciding where to have your TDI executed.

To have TDI executed locally, define the enviromental variable _____ on your MDSplus client machine.

Slicing Arrays

If a data signal has 4 channels multiplexed together, you can isolate the channels as follows (say that \DIVBOLOM_IRRAD is such a channel):

   _c0 = map( \DIVBOLOM_IRRAD, make_range(0,*,4) );
   _c1 = map( \DIVBOLOM_IRRAD, make_range(1,*,4) );
   _c2 = map( \DIVBOLOM_IRRAD, make_range(2,*,4) );
   _c3 = map( \DIVBOLOM_IRRAD, make_range(3,*,4) );

If a 3-dimensional array consists of 65x65 arrays at various times, you could get the 6th frame as follows (IDL is used to invoke the TDI):

   mdsopen,'efit',109070
   d3 = mdsvalue('_d3=\PSIRZ')
   cols = mdsvalue('_COLS=SIZE(_d3,0)')
   rows = mdsvalue('_ROWS=SIZE(_d3,1)')
   d2 = mdsvalue('_d2=set_range( _ROWS, _COLS, map(_d3,make_range(5,*,*)) )')

Some aspects of TDI you should know about

You can not index an array on the left side of an equal sign.

   IDL> print,MDSVALUE("_a[5]=3")
   % MDSVALUE: %TDI$-E-INV_OPC, Invalid operator code in a function
     %TDI Error in EQUALS(_a[5], 3)
     %TDI Error in EVALUATE(_a[5] = 3)
     %TDI Error in EXECUTE("_a[5]=3")

Beware of this parsing "feature:"

        _sum = SUM(_data[0:_nm1]);
     %TREE$-W-NOT_OPEN, tree not currently open
     %TDI Error in COMPILE("\t_sum = SUM(_data[0:_nm1]);")
     %TDI Syntax error near # marked region
             _sum = SUM(_data[0:_n##m1]##);
     %TDI Error in EXECUTE("\t_sum = SUM(_data[0:_nm1]);")

You need a space after the colon when followed by a variable:

          _sum = SUM(_data[0 : _nm1]);

When calling a routine written in another language, data type and memory allocation must be correct. It is best to explicitly type the variables in TDI before making calls to external libraries. E.g.
```
   _inX = D_FLOAT( DATA( DIM_OF( _Sig ) ) ); /* to make double precision */
   _inY = F_FLOAT( DATA( _Sig ) );           /* to make single precision */
    
```

Be aware of integer arithmetic:

       _i = 7/4;	/* will result in _i=1 */

There is no symbolic debugger. Use the WRITE statement:
```
        WRITE(*,'_data=',_data[0 : 10]);
```

In a TDI .FUN file, lines are continued with no special characters:

        _sum = _a + _b 
                  + _c + _d    ;

TDI Reference Pages

See the TDI reference pages for further information

Move up a level