The Xtract Universal server allows accessing meta data and logging information through web calls.

Basic URL #

The basic URL for web calls uses the following format: [protocol]://[host or IP address]:[port]/.
Make sure to use the correct protocol:

Protocol Syntax Example
HTTP http://[host].[domain]:[port] http://todd.theobald.local:8065
HTTP http://[host]:[port] http://localhost:8065
HTTPS https://[host].[domain]:[port] https://todd.theobald.local:8165
Requires a dedicated host name and X.509 certificate, see web server settings.

Hinweis: Make sure to use the correct ports, see Server Ports.

Web calls can be used to:

Query all Logs #

URL Description
http(s)://[host]:[port]/log/?req_type=all Returns all extraction and server logs.

Note: Server logs files are deleted after a defined period of days, see Server Setting - Web Server.

Response

The web call returns the following information:

  • LineCount: row number of the log entry.
  • Name: name of the extraction / name of the server.
  • Timestamp: timestamp of the extraction or server log.
  • State: returns a number between 2 and 4 for a server extraction or the number 5 for a server log, see table below.
  • StateDescr: description of the state, see table below.
  • LogLevel: type of the log (“Error”, “Info” “Warning” or “Debug”).
  • Source technical name of the component that generates the log info.
  • Message: content of the log.
State StateDescr Description
2 Running The extraction is running.
3 FinishedNoErrors Extraction succeeded without errors.
4 FinishedErrors Extraction is finished with at least one error.
5 NotAvailable The status for a server log.

For information on how to read extractions logs, see Extraction Logs.

Example

Example URLExample Response
https://todd.theobald.local:8165/log/?req_type=all 
LineCount,Name,Timestamp,State,StateDescr,LogLevel,Source,Message
1,MAKT,2023-02-20_09:49:23.941,3,FinishedNoErrors,Info,LiveDataExtraction,Product version 5.21.10.14
2,MAKT,2023-02-20_09:49:23.942,3,FinishedNoErrors,Info,LiveDataExtraction,Using Theobald.Extractors Interface
3,MAKT,2023-02-20_09:49:23.999,3,FinishedNoErrors,Info,LiveDataExtraction,Theobald.Extractors version 1.39.3.13
4,MAKT,2023-02-20_09:49:23.999,3,FinishedNoErrors,Info,LiveDataExtraction,Executing Table extraction
...
1,[server],2023-02-20_09:49:10.208,5,NotAvailable,Warning,VersionStore,Configuration was created by a development build. This can lead to unexpected behaviour.
2,[server],2023-02-20_09:49:10.258,5,NotAvailable,Info,AsyncTcpServer,Trying to listen on [::]:8065...
3,[server],2023-02-20_09:49:10.258,5,NotAvailable,Info,AsyncTcpServer,Listening on [::]:8065
4,[server],2023-02-20_09:49:23.353,5,NotAvailable,Info,AsyncTcpServer,Client [::1]:51531 connected
...

Query Logs at Specific Timestamps #

URL Description
http(s)://[host]:[port]/log/?req_type=all&timestamp=[yyyy-MM-dd] Returns all logs of the specified date.
http(s)://[host]:[port]/log/?req_type=all&timestamp=[yyyy-MM-dd_HH:mm:ss.SSS] Returns all logs of the specified timestamp.
http(s)://[host]:[port]/log/?req_type=all&min=[yyyy-MM-dd] Returns all logs after the specified date.
http(s)://[host]:[port]/log/?req_type=all&min=[yyyy-MM-dd_HH:mm:ss.SSS] Returns all logs after the specified timestamp.
http(s)://[host]:[port]/log/?req_type=all&max=[yyyy-MM-dd] Returns all logs before the specified date.
http(s)://[host]:[port]/log/?req_type=all&max=[yyyy-MM-dd_HH:mm:ss.SSS] Returns all logs before the specified timestamp.
http(s)://[host]:[port]/log/?req_type=all&min=[yyyy-MM-dd]&max=[yyyy-MM-dd] Returns all logs between two dates.
http(s)://[host]:[port]/log/?req_type=all&min=[yyyy-MM-dd_HH:mm:ss.SSS]&max=[yyyy-MM-dd_HH:mm:ss.SSS] Returns all logs between two timestamps.
http(s)://[host]:[port]/log/?req_type=all&past_days=[number_of_days] Returns all logs since n days.

Response

The web call returns the following information:

  • LineCount: row number of the log entry.
  • Name: name of the extraction / name of the server.
  • Timestamp: timestamp of the extraction or server log.
  • State: returns a number between 2 and 4 for a server extraction or the number 5 for a server log, see table below.
  • StateDescr: description of the state, see table below.
  • LogLevel: type of the log (“Error”, “Info” “Warning” or “Debug”).
  • Source technical name of the component that generates the log info.
  • Message: content of the log.
State StateDescr Description
2 Running The extraction is running.
3 FinishedNoErrors Extraction succeeded without errors.
4 FinishedErrors Extraction is finished with at least one error.
5 NotAvailable The status for a server log.

For information on how to read extractions logs, see Reading Extraction Logs.

Example

Example URLExample Response
https://todd.theobald.local:8165/log/?req_type=all&past_days=2 
LineCount,Name,Timestamp,State,StateDescr,LogLevel,Source,Message
1,MAKT,2023-02-20_09:49:23.941,3,FinishedNoErrors,Info,LiveDataExtraction,Product version 5.21.10.14
2,MAKT,2023-02-20_09:49:23.942,3,FinishedNoErrors,Info,LiveDataExtraction,Using Theobald.Extractors Interface
3,MAKT,2023-02-20_09:49:23.999,3,FinishedNoErrors,Info,LiveDataExtraction,Theobald.Extractors version 1.39.3.13
4,MAKT,2023-02-20_09:49:23.999,3,FinishedNoErrors,Info,LiveDataExtraction,Executing Table extraction
...
1,[server],2023-02-20_09:49:10.208,5,NotAvailable,Warning,VersionStore,Configuration was created by a development build. This can lead to unexpected behaviour.
2,[server],2023-02-20_09:49:10.258,5,NotAvailable,Info,AsyncTcpServer,Trying to listen on [::]:8065...
3,[server],2023-02-20_09:49:10.258,5,NotAvailable,Info,AsyncTcpServer,Listening on [::]:8065
4,[server],2023-02-20_09:49:23.353,5,NotAvailable,Info,AsyncTcpServer,Client [::1]:51531 connected
...
https://todd.theobald.local:8165/log/?req_type=all&min=2023-02-20_09:49:24.500&max=2023-02-20_09:50:00.000 
LineCount,Name,Timestamp,State,StateDescr,LogLevel,Source,Message
22,MAKT,2023-02-20_09:49:24.500,3,FinishedNoErrors,Debug,TheoReadTableExtractor,Data will be extracted in dialog work process
23,MAKT,2023-02-20_09:49:24.501,3,FinishedNoErrors,Debug,TheoReadTableExtractor,"Fetching packages (50,000 rows per package)"
24,MAKT,2023-02-20_09:49:24.653,3,FinishedNoErrors,Debug,TheoReadTableExtractor,Z_THEO_READ_TABLE version 1.x
25,MAKT,2023-02-20_09:49:24.653,3,FinishedNoErrors,Debug,TheoReadTableExtractor,Received package #1 (1 rows)
26,MAKT,2023-02-20_09:49:24.657,3,FinishedNoErrors,Info,LiveDataExtraction,Starting to write 1 rows to destination...
27,MAKT,2023-02-20_09:49:24.668,3,FinishedNoErrors,Info,LiveDataExtraction,Finished writing rows to destination
28,MAKT,2023-02-20_09:49:24.712,3,FinishedNoErrors,Info,TheoReadTableExtractor,Extraction finished - received 1 rows in total
29,MAKT,2023-02-20_09:49:24.714,3,FinishedNoErrors,Debug,LiveDataExtraction,Writing results to destination completed
6,[server],2023-02-20_09:49:24.802,5,NotAvailable,Debug,ProcessAsync,Theobald.Xu.Web.Worker.exe (16240) exited with 0x0 - The operation completed successfully
7,[server],2023-02-20_09:49:36.257,5,NotAvailable,Info,AsyncTcpServer,Client [::1]:51533 connected
8,[server],2023-02-20_09:49:36.262,5,NotAvailable,Debug,ProcessAsync,Theobald.Xu.Web.Worker.exe (16368) started

Query all Server Logs #

URL Description
http(s)://[host]:[port]/log/?req_type=server Returns a list of timestamps that correspond to server logs.

Response

The web call returns timestamps in the format [yyyy-MM-dd_HH:mm:ss.SSS].
Use the timestamps to query the content of the logs, see Query Server Logs at a Specific Timestamp.

Note: Server log files are deleted after a defined period of days, see Server Setting - Web Server.

Example

Example URLExample Response
https://todd.theobald.local:8165/log/?req_type=server 
Timestamp
2023-02-20_09:49:10.055
2023-02-15_13:49:38.401

Query Server Logs at Specific Timestamps #

URL Description
http(s)://[host]:[port]/log/?req_type=server&timestamp=[yyyy-MM-dd_HH:mm:ss.SSS] Returns the server logs of the specified timestamp.

Tip: Query timestamps that correspond to server logs, see Query all Server Logs.
To query server logs before, after or between timestamps, see Query Logs at Specific Timestamps.

Note: Server log files are deleted after a defined period of days, see Server Setting - Web Server.

Response

The web call returns the following information:

  • LineCount: row number of the log entry.
  • Name: name of the server.
  • Timestamp: timestamp of the server log.
  • State: returns the number 5 to indicate that the log is a server log.
  • StateDescr: description of the state.
  • LogLevel: type of the log (“Error”, “Info” “Warning” or “Debug”).
  • Source technical name of the component that generates the log info.
  • Message: content of the log.

Example

Example URLExample Response
https://todd.theobald.local:8165/?req_type=server&timestamp=2023-02-20_09:49:10.228 
LineCount,Name,Timestamp,State,StateDescr,LogLevel,Source,Message 
1,[server],2023-02-20_09:49:10.258,5,NotAvailable,Info,AsyncTcpServer,Trying to listen on [::]:8065... 
2,[server],2023-02-20_09:49:10.258,5,NotAvailable,Info,AsyncTcpServer,Listening on [::]:8065 
3,[server],2023-02-20_09:49:23.353,5,NotAvailable,Info,AsyncTcpServer,Client [::1]:51531 connected 
4,[server],2023-02-20_09:49:23.372,5,NotAvailable,Debug,ProcessAsync,Theobald.Xu.Web.Worker.exe (16240) started 
5,[server],2023-02-20_09:49:24.802,5,NotAvailable,Debug,ProcessAsync,Theobald.Xu.Web.Worker.exe (16240) exited with 0x0 - The operation completed successfully 
6,[server],2023-02-20_09:49:36.257,5,NotAvailable,Info,AsyncTcpServer,Client [::1]:51533 connected 
7,[server],2023-02-20_09:49:36.262,5,NotAvailable,Debug,ProcessAsync,Theobald.Xu.Web.Worker.exe (16368) started

Query a List of all Defined Extractions #

URL Description
http(s)://[host]:[port]/extractions Returns a list of all defined extractions.
http(s)://[host]:[port]/config/extractions/ Returns a list of all defined extractions with more details and in JSON format.

Response

The web calls return the following information:

  • Name: name of the extraction.
  • Type: extraction type, e.g., Table, Report, ODP, etc.
  • Source: name of the SAP source connection.
  • Destination: name of the destination.
  • LastRun: timestamp of the last execution.
  • RowCount: number of the last extracted data records.
  • LastChange: timestamp of the last change.
  • Created: timestamp of the creation.

Example

Example URLExample Response
https://todd.theobald.local:8165/extractions 
Name,Type,Source,Destination,LastRun,RowCount,LastChange,Created
MAKT,Table,ec5,csv,2022-12-15_13:30:08.921,177318,2023-02-15_13:49:38.401,2022-12-12_08:39:27.407
2LIS,ODP,ec5,AzureStorageAD,2022-08-18_10:55:00.189,59058,2023-01-20_11:26:05.641,2022-08-18_10:46:50.721
COUNTRY,Hierarchy,bw2,http-csv,2022-12-01_12:53:57.098,8,2022-12-01_12:53:53.599,2022-10-05_10:41:43.848
RLT10010,Report,ec5,csv,2023-01-12_11:11:48.975,21,2022-12-13_11:07:36.437,2022-06-30_08:24:47.755
https://todd.theobald.local:8165/config/extractions/ 
{
    "extractions": 
    [
        {
            "name": "MAKT",
            "type": "Table",
            "source": "ec5",
            "destination": "csv",
            "latestRun": {
                "started": "20221215T133008.921Z",
                "duration": "PT00H00M02.850S",
                "rowsCount": 177318,
                "state": "FinishedNoErrors"
            },
            "lastChange": {
                "machine": "SHERRI",
                "user": "alice",
                "timestamp": "20230215T134938.401Z"
            },
            "created": {
                "machine": "SHERRI",
                "user": "alice",
                "timestamp": "20221212T083927.407Z"
            }
        }
    ]
}

Query a Specific Extraction at a Specific Timestamp #

URL Description
http(s)://[host]:[port]/log/?req_type=extraction&name=[extraction_name]&timestamp=[yyyy-MM-dd_HH:mm:ss.SSS] Returns detailed logs of the specified extraction at the specified timestamp.

Tip: Query the timestamp of the last run of an extraction as described in Query a List of all Defined Extractions or query timestamps using the logs as described in Query all Logs.

Response

The web call returns the following information:

  • LineCount: row number of the log entry.
  • Name: name of the extraction.
  • Timestamp: timestamp of the extraction.
  • State: returns a number between 2 and 4 for a server extraction, see table below.
  • StateDescr: description of the state, see table below.
  • LogLevel: type of the log (“Error”, “Info” “Warning” or “Debug”).
  • Source technical name of the component that generates the log info.
  • Message: content of the log.
State StateDescr Description
2 Running The extraction is running.
3 FinishedNoErrors Extraction succeeded without errors.
4 FinishedErrors Extraction is finished with at least one error.

For information on how to read extractions logs, see Extraction Logs.

Example

Example URLExample Response
https://todd.theobald.local:8165/log/?req_type=extraction&name=makt&timestamp=2023-02-20_09:49:23.807 
LineCount,Name,Timestamp,State,StateDescr,LogLevel,Source,Message
1,makt,2023-02-21_11:04:33.765,3,FinishedNoErrors,Info,LiveDataExtraction,Product version 5.21.10.14
2,makt,2023-02-21_11:04:33.766,3,FinishedNoErrors,Info,LiveDataExtraction,Using Theobald.Extractors Interface
3,makt,2023-02-21_11:04:33.817,3,FinishedNoErrors,Info,LiveDataExtraction,Theobald.Extractors version 1.39.3.13
4,makt,2023-02-21_11:04:33.817,3,FinishedNoErrors,Info,LiveDataExtraction,Executing Table extraction
5,makt,2023-02-21_11:04:33.882,3,FinishedNoErrors,Info,LiveDataExtraction,Found license.
6,makt,2023-02-21_11:04:33.930,3,FinishedNoErrors,Debug,R3ConnectorServerWindows,'Use SAPGUI' expert option is disabled
7,makt,2023-02-21_11:04:33.931,3,FinishedNoErrors,Debug,R3ConnectorServerWindows,"Connecting to SAP application server, using Classic RFC SDK"
8,makt,2023-02-21_11:04:33.931,3,FinishedNoErrors,Debug,R3ConnectorServerWindows,"Client '800',  language 'EN'"
9,makt,2023-02-21_11:04:33.931,3,FinishedNoErrors,Debug,R3ConnectorServerWindows,"User ALICE, Password has been provided"
10,makt,2023-02-21_11:04:33.931,3,FinishedNoErrors,Debug,R3ConnectorServerWindows,Using plain authentication
11,makt,2023-02-21_11:04:34.291,3,FinishedNoErrors,Info,R3ConnectorServerWindows,"Connected to SAP host 'sap-erp-as05.example.com', instance 00, release 740, codepage 4103, user 'ALICE'"
...

Query the Extraction Status #

URL Description
http(s)://[host]:[port]/status/?name=[extraction_name]&timestamp=[yyyy-MM-dd_HH:mm:ss.SSS] Returns the status of the specified extraction at the specified timestamp.

Follow the steps below to create a status check routine:

  1. Run your extraction in asynchronous mode using the following URL schema:
    http(s)://[host]:[port]/?name=[extraction_name]&wait=false
    The asynchronous extraction immediately returns an HTTP-response, while the extraction is still running. For more information on how to trigger extractions, see Execute and Automate - Call via Webservice.
  2. Copy the timestamp that is returned in the HTTP-response header and body of the asynchronous extraction.
    Example: X-XU-Timestamp: 2023-01-28_09:58:47.312.
  3. Use the extraction name and the timestamp to query the status of the extraction using the following URL schema:
    http(s)://[host]:[port]/status/?name=[extraction_name]&timestamp=[yyyy-MM-dd_HH:mm:ss.SSS]
  4. The status of an extraction changes in time. Query the status of the extraction in a loop to trigger follow-up actions once the extraction is finished.

Note: Triggering an extraction in asynchronous mode and querying the extraction status is only possible with push-destinations, e.g.s databases or file destinations.

Response

The web call returns one of the following statuses:

State Description
Running The extraction is running.
FinishedNoErrors Extraction succeeded without errors.
FinishedErrors Extraction is finished with at least one error.

Example

Example URLExample Response
https://todd.theobald.local:8165/status/?name=makt&timestamp=2023-02-21_11:06:16.314 
FinishedNoErrors
https://todd.theobald.local:8165/status/?name=mara&timestamp=2023-02-21_13:11:27.327 
FinishedErrors