Appendix E. SMF Information

E.1 IBM FTP-compatible SMF 119 record subtypes

Co:Z SFTP supports recording SMF type 119 records that are compatible with the following IBM FTP records:

Refer to the z/OS Communications Server: IP Programmer's Guide and Reference for complete documentation on FTP SMF type 119 records. Section SMF Record Formats below highlights Co:Z SFTP specific field information.

E.2 New SMF 119 record subtypes

In addition to standard FTP completion/initialization records above, Co:Z SFTP also creates the following SMF 119 record subtypes:

For more information on the Co:Z SFTP specific type 119 records, see section SMF Record Formats.

Note: Record types 100, 101, 194, and 195 are never written as real SMF records

E.3 Enabling SMF recording

In order to enable recording of Co:Z SFTP SMF 119 records, you must:

  1. configure SMF to allow recording these records and subtypes. See z/OS MVS System Management Facilities (SMF) for more information.

  2. permit the users running Co:Z SFTP client or server jobs READ access to the BPX.SMF FACILITY class resource. Alternatively, you may also use type/subtype specific permissions (see next section).

  3. the nosmf configuration option must not be set. See http://coztoolkit.com/docs/sftp/options.html#options_misc for more information.

  4. in order to get accurate local and remote host/port information for client SMF records, the program COZ_HOME/bin/ssh-socket-info is called by Co:Z once the child ssh session is established.

    This program uses the IBM EZBNMIFR network management API, which requires the ssh-socket-info program to be APF authorized. The Co:Z installer will attempt to set the "+a" extattr bit on this program, but will only succeed if the installing userid has READ access to the BPX.FILEATTR.APF SAF resource. If for some reason, this program is not APF authorized, Co:Z SFTP will operate properly, but the SMF socket information will not be accurate in client SMF records.

Using SMF type/subtype specific permissions

Introduced by APAR OA48775, z/OS now allows non-authorized programs to write specific SMF record types/subtypes. This is supported starting with Co:Z SFTP 4.5.0 using the following steps:

  • Permit the users running Co:Z SFTP client or server jobs READ access to BPX.SMF.119.n resource, for n = {3, 70, 192, 193}.

  • The Co:Z SFTP client and server programs must be program controlled. Starting in release 4.5.0, The Co:Z installer will attempt to set the "+p" extattr bit on the Co:Z SFTP client and server programs (cozsftp_cmd and sftp-server) in the install directory.

  • For running the Co:Z SFTP client in batch, you must explicitly mark the COZ.LOADLIB dataset as program controlled. If you are using Co:Z SFTP server user exits, this load library must also be marked as program controlled.

  • The address spaces where you run Co:Z SFTP must remain program-controlled "clean" - in other words, you may not run any non-program controlled commands in the same address space prior to running Co:Z SFTP:

    • For Co:Z SFTP server, do not run any non-program controlled commands in your system or user-level sftp-server.rc scripts. Commands may be run using $(cmd ...) or `cmd ...` or by temorarily using export _BPX_SHAREAS=NO and back to YES around the command, since these will not run in the same address space.

    • For Co:Z SFTP client, watch for commands that might run in the script that you use to invoke the cozsftp command, or in the /etc/profile or $HOME/.profile scripts. Starting in 4.5.0, the sample SFTPPROC will start the z/OS shell in the same address space but with _BPX_SHAREAS=NO. Any commands issued by the profile scripts prior setting _BPX_SHAREAS=YES will run in a separate address space to avoid dirtying the program-controlled environment.

      To diagnose program control issues in client batch jobs, run the step with: ARGS='-LD /bin/sh -Lx' to enable COZBATCH and z/OS shell tracing.

E.4 Using the Real-Time Co:Z SMF Interface

The Co:Z SFTP client and server will also write SMF 119 records to a Unix datagram socket if it is available. By default, the name of the socket is /var/log/cozsftp.smf.sock unless overridden by the SFTP_SMF_SOCK environment variable. This interface is useful in managed file transfer environments that need real-time access to file transfer events. The real-time interface is independent of actual SMF recording - you may use either real SMF recording, the datagram socket, or both.

SMF 119 record subtypes related to interim file transfer logging are only written using the real-time Co:Z SMF Interface. Real-time logging of these records is enabled by setting the option intermlogging=nnnn where nnnn is the interval in seconds. Interim log messages are written during a file transfer. When this feature is enabled and a file transfer is initiated, an initialization record is written at the start of the transfer (subtype 100 by the Co:Z SFTP server, subtype 101 by the Co:Z SFTP client). At the interval specified, interim records (subtype 194 by the Co:Z SFTP server, subtype 195 by the Co:Z SFTP client) are logged capturing the bytes transferred at the time identified in the record header. See Miscellaneous options.

To use this facility, you must write a program that creates this Unix-domain socket and receives datagram messages from it. Each message will be a SMF record image from a Co:Z SFTP client or server running on the same system. A sample C++ program, CoZSmfServer.C, demonstrates how to use this facility. See the documentation and build instructions in $COZ_HOME/samples/smfapi/CoZSmfServer.C. This sample illustrates the following scenarios: consolidation of BPX.SMF authorization to a single job or user, passing of SMF records in real-time to another program, and real-time logging of initialization, interim and completion file transfer SMF records.

E.5 SMF Record Formats

The z/OS Communications Server: IP Programmer's Guide and Reference contains complete documentation on FTP SMF type 119 records. This section highlights Co:Z SFTP specific field information (shown in bold) as well as record formats for Co:Z SFTP type 119 subtypes.

Common Sections

  • TCP/IP identification

    OffsetLengthFormatDescription
    08EBCIDICSystem name
    88EBCIDICSysplex name
    168EBCIDICTCP/IP stack name
    248EBCIDICTCP/IP release identifier. Set to '011100' for V1 Release 11.
    328EBCDICTCP/IP subcomponent. Set to 'SFTPS' (SFTP server) or 'SFTPC' (SFTP client).
    408EBCDICASName
    488EBCDICUserID
    564binaryASID
    601binaryReason. Set to X'08', Event SMF record.
    613binaryreserved

  • FTP security

    OffsetLengthFormatDescription
    01EBCIDICProtection Mechanism. Set to T: TLS.
    11EBCIDICControl Connection Protection Level. Set to P: Private.
    21EBCDICData Connection Protection Level. Set to P: Private.
    31EBCDICLogin Method. Set to P: Password.
    48EBCDICProtocol level. Set to blanks.
    1220EBCDICCipher Specification. Set to blanks.
    324EBCDICProtection buffer size. Set to 0.
    362binaryReserved

Subtype 3 - FTP client transfer completion

  • Self defining section

    The self-defining section identifies 6 triplets, although 7 are allocated. The triplets are:

    • TCP/IP identification

    • FTP client transfer completion

    • FTP client transfer completion associated data set name

    • FTP client SOCKS - triplet set to zero

    • FTP security

    • FTP user name

  • FTP client transfer completion

    Several fields noted below are set from ssh socket information, if available. See section Enabling SMF recording for additional information.

    OffsetLengthFormatDescription
    04EBCIDICFTP command
    44EBCIDICLocal file type
    816binaryRemote IP address (data connection). Set from ssh socket information, if available.
    2416binaryLocal IP address (data connection). Set from ssh socket information, if available.
    402binaryLocal port (data connection). Set from ssh socket information, if available.
    422binaryRemote port (data connection). Set from ssh socket information, if available.
    4416binaryRemote IP address (control connection). Set equal to the data connection value.
    6016binaryLocal IP address (control connection). Set equal to the data connection value.
    762binaryRemote port (control connection). Set equal to the data connection value.
    782binaryLocal port (control connection). Set equal to the data connection value.
    808EBCIDICServer user id
    888EBCIDICLocal user id
    961EBCIDICData format
    971EBCIDICTransfer mode
    981EBCIDICStructure
    991EBCIDICData set type
    1004binaryTransfer start time
    1044packedTransfer start date
    1084binaryTransfer end time
    1124packedTransfer end date
    1164binaryTransfer duration
    1208binaryTransmission byte count
    1284EBCIDICLast server reply
    1328EBCIDICPDS member name
    1408EBCIDICHost name
    1488EBCIDICAbnormal end information
    1568floating point hexTransmission byte count (float)
    1644binaryTCP connection ID (control connection). Set from ssh socket information, if available.
    1684binaryTCP connection ID (data connection). Set equal to the control connection value.

Subtype 70 - FTP server transfer completion

  • Self defining section

    The self-defining section identifies 6 triplets, although 7 are allocated. The triplets are:

    • TCP/IP identification

    • FTP server transfer completion

    • FTP server host name

    • FTP server first associated data set name

    • FTP server second associated data set name

    • FTP security

  • FTP server transfer completion

    OffsetLengthFormatDescription
    01binaryFTP operation
    13binaryreserved
    44EBCIDICFTP command
    84EBCIDICLocal file type
    1216binaryRemote IP address (data connection)
    2816binaryLocal IP address (data connection)
    442binaryLocal port (data connection)
    462binaryRemote port (data connection)
    4816binaryRemote IP address (control connection). Set equal to the data connection value.
    6416binaryLocal IP address (control connection). Set equal to the data connection value.
    802binaryRemote port (control connection). Set equal to the data connection value.
    822binaryLocal port (control connection). Set equal to the data connection value.
    848EBCIDICClient user id on server
    921EBCIDICData type
    931EBCIDICTransmission mode
    941EBCIDICData Structure
    951EBCIDICData set type
    964binaryTransfer start time
    1004packedTransfer start date
    1044binaryTransfer end time
    1084packedTransfer end date
    1124binaryTransfer duration
    1168binaryTransmission byte count
    1244EBCIDICLast reply to client
    1288EBCIDICPDS member name
    1368EBCIDICAbnormal end information
    1448EBCIDICSecond PDS member name
    1528floating point hexTransmission byte count (float)
    1604binaryTCP connection ID (control connection). Set to 0.
    1644binaryTCP connection ID (data connection). Set to 0.
    16815EBCIDICSession id. Set to a generated value: jobname followed by the last five digits of the process id.
    1831binaryreserved

Subtype 100 - FTP server transfer initialization (real-time SMF data NMI record format)

Real-time transfer SMF records are not written by default. Refer to Miscellaneous options for information on setting the interimlogging option to enable this feature. Additionly, see Using the real-time Co:Z SMF interface for information on accessing real-time SMF records.

  • Self defining section

    The self-defining section identifies 6 triplets, although 7 are allocated. The triplets are:

    • TCP/IP identification

    • FTP server transfer initialization

    • FTP server host name

    • FTP server first associated data set name

    • FTP server second associated data set name

    • FTP security

  • FTP server transfer initialization

    OffsetLengthFormatDescription
    01binaryFTP operation
    11binaryPassive or active mode data connection. Set to X'00': Active using default IP and port.
    22binaryreserved
    44EBCIDICFTP command
    84EBCIDICLocal file type
    1216binaryRemote IP address (data connection)
    2816binaryLocal IP address (data connection)
    442binaryLocal port (data connection)
    462binaryRemote port (data connection)
    4816binaryRemote IP address (control connection). Set equal to the data connection value.
    6416binaryLocal IP address (control connection). Set equal to the data connection value.
    802binaryRemote port (control connection). Set equal to the data connection value.
    822binaryLocal port (control connection). Set equal to the data connection value.
    848EBCIDICClient user id on server
    921EBCIDICData type
    931EBCIDICTransmission mode
    941EBCIDICData Structure
    951EBCIDICData set type
    964binaryData connection start time. Set to the start time of the session.
    1004packedData connection start date. Set to the start date of the session.
    1044binaryControl connection start time. Set equal to the data connection value.
    1084packedControl connection start date. Set equal to the data connection value.
    1128EBCIDICPDS member name
    1208EBCIDICSecond PDS member name
    1284binaryTCP connection ID (control connection). Set to 0.
    1324binaryTCP connection ID (data connection). Set to 0.
    13615EBCIDICSession id. Set to a generated value: jobname followed by the last five digits of the process id.
    1511binaryreserved

Subtype 101 - FTP client transfer initialization (real-time SMF data NMI record format)

Real-time transfer SMF records are not written by default. Refer to Miscellaneous options for information on setting the interimlogging option to enable this feature. Additionly, see Using the real-time Co:Z SMF interface for information on accessing real-time SMF records.

  • Self defining section

    The self-defining section identifies 6 triplets, although 7 are allocated. The triplets are:

    • TCP/IP identification

    • FTP client transfer initialization

    • FTP client associated data set name

    • FTP client SOCKS - triplet set to zero

    • FTP security

    • FTP user name

  • FTP client transfer initialization

    Several fields noted below are set from ssh socket information, if available. See section Enabling SMF recording for additional information.

    OffsetLengthFormatDescription
    04EBCIDICFTP command
    44EBCIDICLocal file type
    816binaryRemote IP address (data connection). Set from ssh socket information, if available.
    2416binaryLocal IP address (data connection)Set from ssh socket information, if available.
    402binaryLocal port (data connection)Set from ssh socket information, if available.
    422binaryRemote port (data connection)Set from ssh socket information, if available.
    4416binaryRemote IP address (control connection). Set equal to the data connection value.
    6016binaryLocal IP address (control connection). Set equal to the data connection value.
    762binaryRemote port (control connection). Set equal to the data connection value.
    782binaryLocal port (control connection). Set equal to the data connection value.
    808EBCIDICServer user id
    888EBCIDICLocal user id
    961EBCIDICData format
    971EBCIDICTransfer mode
    981EBCIDICStructure
    991EBCIDICData set type
    1004binaryStart time of data connection. Set to the start time of the session.
    1044packedStart date of data connection. Set to the start date of the session.
    1084binaryStart time of control connection. Set equal to the data connection value.
    1124packedStart date of control connection. Set equal to the data connection value.
    1168EBCIDICPDS member name
    1241EBCIDICPassive or active mode data connection. Set to X'00': Active using default IP and port.
    1253binaryreserved
    1284binaryTCP connection ID (control connection). Set from ssh socket information, if available.
    1324binaryTCP connection ID (data connection). Set equal to the control connection value.

Subtype 192 - Co:Z SFTP server log messages

  • Self defining section

    The self-defining section identifies 3 triplets, although 7 are allocated. The triplets are:

    • TCP/IP identification

    • Socket connection

    • Co:Z SFTP messages

  • Socket connection

    OffsetLengthFormatDescription
    016binaryRemote IP address
    1616binaryLocal IP address
    322binaryRemote port number
    342binaryLocal port number
    3615EBCDICFTP session ID. Set to a generated value: jobname followed by at most the last five digits of the process id.
    511binaryreserved

  • Co:Z SFTP messages

    This section contains Co:Z SFTP messages, informational level or above, that were associated with the previous transfer. One or more message sub-sections may be included, each with the following layout:

    OffsetLengthFormatDescription
    04binaryTime (in local time)
    44PackedDate (in local time)
    82binaryLength of message that follows
    10variableEBCDICMessage text

Subtype 193 - Co:Z SFTP client log messages

  • Self defining section

    The self-defining section identifies 3 triplets, although 7 are allocated. The triplets are:

    • TCP/IP identification

    • Socket connection

    • Co:Z SFTP messages

  • Socket connection

    Fields noted below are set from ssh socket information, if available. See section Enabling SMF recording for additional information.

    OffsetLengthFormatDescription
    016binaryRemote IP address. Set from ssh socket information, if available.
    1616binaryLocal IP address. Set from ssh socket information, if available.
    322binaryRemote port number. Set from ssh socket information, if available.
    342binaryLocal port number. Set from ssh socket information, if available.
    3615EBCDICFTP session ID. Set to blank.
    511binaryreserved

  • Co:Z SFTP messages

    This section contains Co:Z SFTP messages, informational level or above, that were associated with the previous transfer. One or more message sub-sections may be included, each with the following layout:

    OffsetLengthFormatDescription
    04binaryTime (in local time)
    44PackedDate (in local time)
    82binaryLength of message that follows
    10variableEBCDICMessage text

Subtype 194 - Co:Z SFTP server interim transfer (real-time Co:Z SMF interface)

Real-time transfer SMF records are not written by default. Refer to Miscellaneous options for information on setting the interimlogging option to enable this feature. Additionly, see Using the real-time Co:Z SMF interface for information on accessing real-time SMF records.

  • Self defining section

    The self-defining section identifies 7 triplets. The triplets are:

    • TCP/IP identification

    • FTP server transfer initialization - Set equal to FTP server transfer initialization (subtype 100)

    • FTP server host name

    • FTP server first associated data set name

    • FTP server second associated data set name

    • FTP security

    • FTP interim transfer

  • FTP interim transfer section

    OffsetLengthFormatDescription
    08binaryEstimated file size (bytes). Set to -1 on put (write) or if read and source file size is unknown.
    88binaryEstimated file size (bytes float). Set to -1 on put (write) or if read and source file size is unknown.
    168binaryInterim transmission byte count
    248floating point hexInterim transmission byte count (float)

Subtype 195 - Co:Z SFTP client interim transfer (real-time Co:Z SMF interface)

Real-time transfer SMF records are not written by default. Refer to Miscellaneous options for information on setting the interimlogging option to enable this feature. Additionly, see Using the real-time Co:Z SMF interface for information on accessing real-time SMF records.

  • Self defining section

    The self-defining section identifies 7 triplets. The triplets are:

    • TCP/IP identification

    • FTP client transfer initialization - Set equal to FTP client transfer initialization (subtype 101)

    • FTP client associated data set name

    • FTP client SOCKS - triplet set to zero

    • FTP security

    • FTP user name

    • FTP interim transfer

  • FTP interim transfer section

    OffsetLengthFormatDescription
    08binaryEstimated file size (bytes). Set to -1 on get (write) or if read and source file size is unknown.
    88binaryEstimated file size (bytes float). Set to -1 on get (write) or if read and source file size is unknown.
    168binaryInterim transmission byte count
    248floating point hexInterim transmission byte count (float)


Saint Charles, Missouri
info@coztoolkit.com
+1 636.300.0901

Copyright© 2009 - 2023 Dovetailed Technologies, LLC. All rights reserved. Co:Z® is a registered trademark and Co:Z Toolkit™ is a trademark of Dovetailed Technologies, LLC.

Saint Charles, Missouri
info@coztoolkit.com
+1 636.300.0901

Copyright© 2009 - 2023 Dovetailed Technologies, LLC. All rights reserved. Co:Z® is a registered trademark and Co:Z Toolkit™ is a trademark of Dovetailed Technologies, LLC.