Build an interface
Interfaces, or interface declarations are flow based programs running on the app, typically connecting two systems, but can do so much more.
Interface declaration consists an input module and one or more steps.
Input gets an incoming file or request, parses it and forwards the parsed data to its output. Steps process data from their input and forward the processed data to their output, except when the condition field forbids this, in that case the input is blindly copied to the output without any processing.
Input
An interface has to have at least one input. This input can be one of the following:
Cron
File
Watch folder for new files, move them after parsing to a given archive folder, or delete them.
FTPS Poll
Periodically log in and fetch files from an FTP or FTPs server's folder, move the files after parsing to a given archive folder on the server or delete them.
HTTP Listen
Create a web server and process requests or act as a static webserver serving files from a given folder.
HTTP Poll
Periodically fetch a HTTP endpoint (REST API, HTML page, RSS feed or any other content), can use any HTTP method, for example can be used to do triggering the process and do the authentication in one step.
SFTP Poll
Periodically fetch files from an SFTP server's folder, move the files to a given archive folder on the server or delete them.
SQL Poll
Run SQL select periodically
Parsing input
Every input module apart from cron, and many other steps will have a Parser section in their settings form. This is standardised to make it easier to define behaviour and switching from one input method to another will not affect the other parts of the interface.
none parser
No parsing at all, the whole file or request content will be in the payload without any change.
folder
This is not a real parser, it's a static web server providing content to the HTTP Listen step. This might look strange here, and it is, honestly. This is due to historic reasons, we might move it to the HTTP Listen step later.
xpath
XPath can generate multiple parsed payloads from one file or request. It is not a complete xpath implementation but strong enough for every-day use.
csv parser
Parsing CSV files into a map / dictionary structure.
Please remember, parsed structures are data structures and not JSON, we are only using JSON as representation of the data.
csv; parser
Same as CSV parser but for semicolon separated files.
csv/no-header parser
This is good if there is no header information in the CSV or it's not reliable enough for parsing. Returns a flat structure, but already separated by new lines.
csv/no-header; parser
Same as csv/no-header but with semicolons.
xml parser
Returns XML parsed content of the file, preserving attribute values as - (hyphen) prefixed keys.
json parser
Parsing a JSON string into the underlying data structures the other steps will be able to work from.
_ (underscore) parser
Underscore separated CSV. We have a lovely client who needed this.
bfck parser
Same here as above, it's not really the real BFCK programming language, but the rules in parsing these files were quite close to that.
form-data parser
Parsing application/x-www-form-urlencoded values into the underlying data structures the other steps will be able to work from.
Steps
As mentioned before, steps process data from their input and forward the processed data to their output, except when the condition field forbids this, in that case the input is blindly copied to the output without any processing.
The contition field is in every step's settings form. Condition tester can be used to understand and double-check the type-related quirks.
See Condition field in the form below, and condition tester link on top of it:
Combine Items
Groups payload by given key into a map / dictionary / associative array, for example group records by SKU.
Command
Runs a shell script or other command.
Delay
Delay will introduce an artificial delay into the flow – sometimes this is necessary as Apio is really fast, certain APIs don't like this.
The only option is to set a delay in seconds, but we can also add a condition field, as for all steps.
File
Writes the result of the template expression into a new file.
FTPS Client
Logs in to an FTP or FTPs server, creates a file with the provided filename and content.
HTTP
Creates a HTTP request to an URL and processes the result.
The Ignore Error step will continue processing to the next step even after non-200 response.
Result will be stored in the header field identified by the Key field's value, an extra header field will be added for status code, so error checking is easier. The extra field's name will be '_status' added to the Key field's value.
If
Container step, runs the enclosed steps only if the condition is true.
Javascript
Powerful Javascript step, allowing transparent modification of the header and payload (no need to import or return). By default includes underscore.js functionality for easier data transformation.
Uses otto Javascript engine, not fully compatible with ECMA5 specification. ES6 features are not supported.
Jump
Forwards the payload to the given step's input queue – "re-running" the interface code from that point. Be careful with infinite loops, always use and test conditions!
Loop
Container step, running the enclosed steps until the condition is true. Be careful with infinite loops, always use and test conditions!
M2 Price
Magento 2 Price update step, writing Magento 2 database directly for increased performance. Handle with care, doesn't like customised systems.
M2 Product
Magento 2 Product creation/update step, writing Magento 2 database directly for increased performance. Handle with care, doesn't like customised systems.
M2 Stock
Magento 2 Stock update step, writing Magento 2 database directly for increased performance. Handle with care, doesn't like customised systems.
Paginate
Pagination will break a payload into several smaller payload without changing the underlying structure or surrounding content.
Parse
Response
Return with a HTTP status code and a body to HTTP Listen, File, FTPs or other supporting input step. If defined, the input step will wait for the Response step to be reached (until timeout from the input step) and return the response values, or as File, FTPs and other non-request-response style inputs, will move files from Archive to the Errors folder for 400+ response codes.
Send Data
Send Data step is to send the whole payload or part of it to the Apio dashboard. Steps send data by default, with this step we can send targeted information.
Set
This step will set a header field to the result of the given template expression. Very useful for situations when speed and memory consumption is key.
SFTP Client
Logs in to an SFTP server and creates a file with the result of the template expression.
SQL
Currently MySQL is supported.
SQLite
Reads or writes an SQLite database file. Table and column creation is automatic.
Status
Sets the status for the current payload on the dashboard and optionally stops the flow.
Stop
Stops the flow from further processing.