Home > Coding > Fun with FTP

Fun with FTP

The z/OS FTP server is an often overlooked component of the z/OS environment. As well as supporting all the usual commands, it has some facilities that are specific to the z/OS environment. This means that when used in conjunction with a suitable client you can create some pretty nifty tools. For the HTML Application (HTA) I wrote recently I used the Windows ftp client and some VBScript to create a Windows desktop file transfer interface.

This particular application needed to transfer several XMIT files (output files from a TSO SEND command) to a host and then restore the original files from those datasets.

The Windows ftp client is a simple command line client although you can pre build the commands by putting them into a file and then pointing the ftp command at that file. By running it under Windows cmd.exe program it is also possible to pipe the output into another file that you can then analyze to check the results.
When run in this mode, the Windows ftp client is not really ‘interactive’ in that it just keeps issuing the commands regardless but by splitting the input up into multiple ftp sessions and using the (in this case) scripting environment on the Windows machine it is possible to create semi automated ftp applications.

This is the VBScript function I am using to run the ftp program:

Function FTP(fni, fno)
CreateObject(“WScript.Shell”).Run “%comspec% /k ftp -n -s:” & chr(34) & fni & chr(34) & ” > ” & chr(34) & fno & chr(34) & ” & exit” , 0 , true
End Function

  • %compec% is the Windows environment variable to cmd.exe
  • /k tells cmd.exe to wait until the program it is running (ftp in this case) ends before it ends
  • ftp is the program I want to run
  • -n suppresses the automated login and lets the input file drive the login
  • -s:chr(34) & fni & chr(34) specifies the input file name, enclosed in double quotes (chr(34)) in case it has any embedded spaces.
  • > pipes the output into the named file
  • & exit tells cmd.exe to end when the ftp command is done
  • 0 (zero) tells the scripting engine to hide the cmd.exe window
  • true tells the script to wait for the program it is running (cmd.exe) to end before continuing.

The above settings cause the ftp command to run synchronously with the script and also hides the cmd.exe window so the user never sees it.

Sending Files to the Host

XMIT file records are fixed length 80 characters long, therefore when we store them on the host they must be stored with the same format otherwise the receive is not going to work.
This is where the SITE command comes in.

The SITE command lets you set defaults on the host for the ftp session that it will use for the following commands. In this case I wanted to tell it to create the files as Fixed Block files with a record length of 80 bytes.
So in my ftp session stream I included the following:

QUOTE SITE FILETYPE=SEQ LRECL=80 BLKSIZE=3200   RECFM=FB CYLINDERS PRIMARY=1 SECONDARY=1
BIN

  • FILETYPE=SEQ tells the ftp server to store files a sequential files
  • LRECL=80 BLKSIZE=3200 RECFM=FB tells it the create the new files with a record length of 80, a block size of 3200 and that the files should be fixed blocked.
  • CYLINDERS PRIMARY=1 SECONDARY=1 specifies the default space allocations to use (my files are not very big so this will always be sufficient in my case).
  • BIN tells ftp to transfer the files as binary so that it does not try to convert the data from ascii to EBCDIC.

Analyzing the FTP Output

Although the ftp client program does not produce any output directly,  because I am running under the cmd.exe program I can pipe the session output (the stuff that would normally appear in the dos window) to a file.
I can then use the scripting language to go through the output file to see if everything worked, or if something went wrong.

This is the output from a session where I have no connection to the host

ftp> Unknown host systemz.ibm.com.
ftp> OPEN systemz.ibm.com
Not connected.
ftp> USER AA
Invalid command.
ftp> ***
Not connected.
ftp> QUOTE SITE FILETYPE=SEQ LRECL=80 BLKSIZE=3200 RECFM=FB CYLINDERS PRIMARY=1 SECONDARY=1
Not connected.
ftp> BIN
Not connected.
ftp> DELETE ‘AA.class.uss.xmit’
Not connected.
ftp> PUT “C:\zprof\class.uss.xmit” ‘AA.class.uss.xmit’
QUIT

You can see that the first line says “ftp> Unknown host …”. It’s quite easy to have your script go through the output looking for this type of error. What happens then is up to you of course,  but when I detect an error I display the ftp output file on the user screen in the web page.
For example, this is the ftp output when a file is successfully transferred:

ftp> PUT “C:\zprof\class.source.xmit” ‘DELLI2.class.source.xmit’
200 Port request OK.
125 Storing data set DELLI2.CLASS.SOURCE.XMIT
250 Transfer completed successfully.

Notice that a line starting with a 250 code occurs a few lines after the put command. Thus by looking for the ‘ftp > PUT” command and then searching for a 250 line I can tell that the put worked ok.

In contrast, this is one that failed:

ftp> PUT C:\zprof\xxx.thisisbad.class.samplib.xmit” ‘DELLI2.xxx.thisisbad.class.samplib.xmit’
200 Port request OK.
501 Invalid data set name “‘DELLI2.xxx.thisisbad.class.samplib.xmit'”. Use MVS Dsname conventions.
ftp> PUT “C:\zprof\class.source.xmit” ‘DELLI2.class.source.xmit’

In this case, there is no 250 line. The error is caused by a bad dataset name and I could use the 501 code to detect it but the error might not always be a 501. Instead what I did was look for the next put command. If I get another PUT command before I find a 250 response I know the previous put failed.

Job Submission

One of the cool things you can do with the z/OS ftp server is submit jobs to the Job Entry Subsystem (JES). The process is pretty simple:

Put your JCL into a file
Create the ftp control stream
Run the ftp command

The jcl is standard z/OS JCL, the ftp server does not validate it in any way. The only thing to watch out for is the job name that you use. The z/OS ftp server can be configured (by the systems programmer) to support differing JES interface levels. The simplest level, JESINTERFACELEVEL 1 requires that the job name of any job submitted by the ftp server match the submitting users userid, or consist of the userid plus one character.
Thus if my userid is ELLIS say, then any jobs I submit must have a job name of the form ELLIS or ELLIS1, ELLIS2, ELLISA, ELLISZ etc.
JESINTERFACELEVEL 2 is more flexible but unless you are developing for your own systems and can guarantee the JES interface level that will be available (I wasn’t) then you need to either enforce the above rule or enable the user to easily customize the job name.

To submit the jcl from the file you use the following two commands in the ftp input stream:

QUOTE SITE FILETYPE=JES
PUT filename

The SITE command tells the z/OS ftp server to direct the incoming requests to JES.
The PUT commands sends the pre built jcl from the file you put it in, to the z/OS ftp server, that then submits it to JES.

This is some typical output from a job submission session:

ftp> QUOTE SITE FILETYPE=JES
200 SITE command was accepted
ftp> PUT C:\JCL.txt
200 Port request OK.
125 Sending Job to JES internal reader FIXrecfm 80
250-It is known to JES as JOB02495
250 Transfer completed successfully.
ftp: 2954 bytes sent in 0.01Seconds 295.40Kbytes/sec.ftp> QUIT

The interesting line here is the 250 line. This line contains the job number of the job I just submitted which, in this case is JOB02495.

On its own, this is not very exciting. I could logon to the target z/OS system and go and look at the job, but I can do better than that.

If I use the ftp client to send the following commands to the z/OS ftp server I can find out what state the job is in, INPUT (waiting to execute), ACTIVE (Executing) or OUTPUT (ended):

QUOTE SITE FILETYPE=JES
DIR JOB02495 C:\JOBSTATUS.txt

The DIR command will return information about the job in the C:\JOBSTATUS.txt file.
Again, I can then use the scripting language to analyze that output to determine what is happening with the job and display that information to the user.

What I did was create a loop, that issued the above ftp commands every few seconds and display the status of the job to the user. It exited the loop once the job was on the output queue.

What I have found is that the format of the output returned by the DIR command can vary from system to system. For example for the input queue I have seen:

ELLIS1  JOB02495 ELLIS   INPUT  A
ELLIS1  JOB02495 INPUT

For the executing queue:

ELLIS1  JOB02495 ELLIS ACTIVE A
ELLIS1  JOB02495 ACTIVE

For the output queue:

ELLIS1  JOB02495 OUTPUT 0 Spool Files
ELLIS1  JOB02495 ELLIS  OUTPUT A        RC=0000
ELLIS1  JOB02495 ELLIS  OUTPUT A        RC=0012
ELLIS1  JOB02495 ELLIS  OUTPUT A        RC=(JCL Error)

AS you can see, the format and information contained varies considerably. However it is possible to at least consistently detect the state of the job, INPUT, ACTIVE or OUTPUT. Rather than look for one of those words in a specific position in the output lines I split the lines up into word tokens and then just go through the tokens looking for the particular values to detect the job’s state.

The main issue with the differing formats is the lack of return code or error information in this particular format of status data that is returned:

ELLIS1  JOB02495 OUTPUT 0 Spool Files

When this format was returned, regardless of the state of the job, this is all I ever got back. Remember, these are not my systems so I cannot change the output format. This was a problem because I wanted to be able to tell the user what happened to the job without them having to logon on the z/OS system in order to find out.

Obtaining the Job Output

Another neat feature of the z/OS FTP server is that you can use it to obtain job output. There is one requirement though and that is that the job output MUST be on a HELD queue.

The following commands will obtain the ALL the job output for the specified job and store it in a a file:

QUOTE SITE FILETYPE=JES
QUOTE SITE JESJOBNAME=*
QUOTE SITE JESJOWNER=*
QUOTE SITE JESSTATUS=ALL
GET JOB02495 C:\JOBOUTPUT.txt

The GET command will obtain the job output for the specified job and store it in the C:\JOBOUTPUT.txt file. The JESJOBNAME=*, JESJOWNER=* and JESSTATUS=ALL commands just ensure that any default setting in the ftp server do not prevent me from obtaining all my output. They have no effect on a system defined with JESINTERFACELEVEL 1 but do no harm.

What you get back is the job output. You can then use your scripting environment to analyze the output and or display it to the user. Remember previously that in some instances when obtaining the job status I was not getting any valid return code information in the status line when the job had ended. What I did was go through the output looking for specific messages such as JCL error or abend messages. I also looked at the return code message (IEF142I) for each step to determine the highest return code returned by any step in the job. If I found any error messages or a non zero return code I displayed an appropriate message to the user and displayed the job output on the web page so that they could examine it.

Advertisements
Categories: Coding
  1. No comments yet.
  1. No trackbacks yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: