Archiver Plugins¶
Overview¶
Archiver plugins are used for retrieving or saving scanned payloads. A payload can be anything from the initial payload scanned, or extracted payloads from previous scans. There are two types of archivers, source and destination.
destination¶
Archiver plugins used as a destination useful for saving payloads, be it the original scanned payload or any extracted payloads. Multiple destination archivers can be defined, allowing for a payload to be saved in either a single or multiple locations. The results from this plugin method may be used to subsequently load the payload again.
Destination archiver plugins can be defined multiple ways. In these examples, we will
use the filedir
archiver plugin.
From stoq.cfg
:
[core]
dest_archivers = filedir
Note
Multiple plugins can be defined separated by a comma
From the command line:
$ stoq run -A filedir [...]
Note
Multiple plugins can be defined by simply adding the plugin name
Or, when instantiating the Stoq()
class:
>>> import stoq
>>> dest_archivers = ['filedir']
>>> s = Stoq(dest_archivers=dest_archivers, [...])
source¶
Archiver plugins used as a source retrieve payloads for scanning. This is useful in several use cases, such as when using a provider plugin that isn’t able to pass a payload to stoQ. For example, if the provider plugin being used leverages a queueing system, such as RabbitMQ, there may be problems placing multiple payloads onto a queue as it is inefficient, prone to failure, and does not scale well. With archiver plugins as a source, the queuing system can be leveraged by sending a message with a payload location, and the archiver plugin can then retrieve the payload for scanning. The ArchiverResponse results returned from ArchiverPlugin.archive() is used to load the payload.
Source archiver plugins can be defined multiple ways. In these examples, we will
use the filedir
archiver plugin.
From stoq.cfg
:
[core]
source_archivers = filedir
Note
Multiple plugins can be defined separated by a comma
From the command line:
$ stoq run -S filedir [...]
Note
Multiple plugins can be defined by simply adding the plugin name
Or, when instantiating the Stoq()
class:
>>> import stoq
>>> source_archivers = ['filedir']
>>> s = Stoq(source_archivers=source_archivers, [...])
Writing a plugin¶
Unlike most other stoQ plugins, archiver plugins have two core methods, of which at least one of the below is required.
- archive
- get
The archive
method is used to archive payloads that are passed to stoQ or extracted
from other plugins. In order for a payload to be archived, that attribute should_archive
must be set to True
in the payloads PayloadMeta
object. If set to False
, the
payload will not be archived.
An archiver plugin must be a subclass of the ArchiverPlugin
class.
As with any plugin, a configuration file must also exist and be properly configured.
Example¶
from typing import Dict, Optional
from configparser import ConfigParser
from stoq.plugins import ArchiverPlugin
from stoq.data_classes import ArchiverResponse, Payload, RequestMeta, PayloadMeta
class ExampleArchiver(ArchiverPlugin):
def __init__(self, config: ConfigParser, plugin_opts: Optional[Dict]) -> None:
super().__init__(config, plugin_opts)
self.archive_path = config.get(
'options', 'archive_path', fallback='/tmp/archive_payload')
def archive(
self, payload: Payload, request_meta: RequestMeta
) -> Optional[ArchiverResponse]:
with open(f'{self.archive_path}', 'wb) as out:
out.write(payload.content)
ar = ArchiverResponse({'path': f'{self.archive_path}'})
return ar
def get(self, task: ArchiverResponse) -> Optional[Payload]:
with open(task.results['path'], 'rb') as infile:
return Payload(
infile.read(),
PayloadMeta(
extra_data={'path': task.results['path']}))
Note
ArchiverPlugin.archive() returns an ArchiverResponse object, which contains metadata that is later used by ArchiverPlugin.get() to load the payload.
API¶
-
class
stoq.plugins.archiver.
ArchiverPlugin
(config, plugin_opts)[source]¶ -
archive
(payload, request_meta)[source]¶ Archive payload
Parameters: - payload (
Payload
) – Payload object to archive - request_meta (
RequestMeta
) – Originating request metadata
Return type: Optional
[ArchiverResponse
]Returns: ArchiverResponse object. Results are used to retrieve payload.
>>> from stoq import Stoq, Payload >>> payload = Payload(b'this is going to be saved') >>> s = Stoq() >>> archiver = s.load_plugin('filedir') >>> archiver.archive(payload) ... {'path': '/tmp/bad.exe'}
- payload (
-
get
(task)[source]¶ Retrieve payload for processing
Parameters: task ( ArchiverResponse
) – Task to be processed to load payload. Must contain ArchiverResponseresults from ArchiverPlugin.archive()
Return type: Optional
[Payload
]Returns: Payload object for scanning >>> from stoq import Stoq >>> s = Stoq() >>> archiver = s.load_plugin('filedir') >>> task = ArchiverResponse(results={'path': '/tmp/bad.exe'}) >>> payload = archiver.get(task)
-
Response¶
-
class
stoq.data_classes.
ArchiverResponse
(results=None, errors=None)[source]¶ Object containing response from archiver destination plugins
Parameters: - results (
Optional
[Dict
[~KT, ~VT]]) – Results from archiver plugin - errors (
Optional
[List
[str
]]) – Errors that occurred
>>> results = {'file_id': '12345} >>> archiver_response = ArchiverResponse(results=results)
- results (