1.6.2: Update cryptography (42.0.0) and Flask (3.0.0) dependencies
1.6.1: Update cryptography and Flask dependency and deprecate Python 3.7
1.6.0: Automatically generate am .mtz
for your local transforms
1.5.2: Add logging output for invalid / missing params in xml serialization
1.5.1: Add ignored files to starter and use README for pypi
1.5.0: XML Serialization via ElementTree
instead of string interpolation
1.4.4: Added skeletons for csv export in template dir and made project.py application import compatible with docs
1.4.0 + 1.4.1: Both versions are incompatible with python3.7 and lower.
1.4.2: Fixed python3.6 incompatibility
Note: Support for Python 2 has been officially discontinued as of July 2021. Please use Python 3.8 or higher to use up-to-date versions of Maltego TRX.
To install the trx library run the following command:
pip install maltego-trx
After installing, you can create a new project by running the following command:
maltego-trx start new_project
This will create a folder new_project with the recommended project structure.
If you want to copy the starter files to your current directory, run the following command:
maltego-trx init
Alternatively, you can copy either the gunicorn
or apache
example projects from the demo
directory. These also
include Dockerfile and corresponding docker-compose configuration files for production deployment.
Adding a Transform:
Add a new transform by creating a new python file in the "transforms" folder of your directory.
Any file in the folder where the class name matches the filename, and the class inherits from Transform, will automatically be discovered and added to your server.
A simple transform would look like the following:
new_project/transforms/GreetPerson.py
from maltego_trx.entities import Phrase
from maltego_trx.transform import DiscoverableTransform
class GreetPerson(DiscoverableTransform):
"""
Returns a phrase greeting a person on the graph.
"""
@classmethod
def create_entities(cls, request, response):
person_name = request.Value
response.addEntity(Phrase, "Hi %s, nice to meet you!" % person_name)
You can start the development server, by running the following command:
python project.py runserver
This will start up a development server that automatically reloads every time the code is changed.
You can run a gunicorn transform server, after installing gunicorn on the host machine and then running the command:
gunicorn --bind=0.0.0.0:8080 --threads=25 --workers=2 project:application
For publicly accessible servers, it is recommended to run your Gunicorn server behind proxy servers such as Nginx.
The demo
folder provides an example project. The Docker files given can be used to set up and run your project in
Docker.
The Dockerfile and docker-compose file can be used to easily set up and run a development transform server.
If you have copied the docker-compose.yml
, Dockerfile
and prod.yml
files into your project, then you can use the
following commands to run the server in Docker.
Run the following to start the development server:
docker-compose up
Run the following command to run a production gunicorn server:
docker-compose -f prod.yml up --build
For publicly accessible servers, it is recommended to run your Gunicorn server behind proxy servers such as Nginx.
Transforms written using this library can be used as either local or server transforms.
To run a local transform from your project, you will need to pass the following arguments:
project.py local <transform_name>
You can find the correct transform_name to use by running python project.py list
.
The following values are not passed to local transforms, and will have dummy values in their place:
type
: local.Unknown
weight
: 100slider
: 100transformSettings
: {}The Transform Registry enables you to annotate Transforms with metadata like display name, description, input and output entities as well as settings. The Transform Registry will automatically generate CSV files that you can import into the pTDS and/or your iTDS.
You can configure your registry with all the info you would normally add for every transform/seed on a TDS. We recommend
creating your registry in an extra file, traditionally called extensions.py
, to avoid circular imports.
# extensions.py
from maltego_trx.decorator_registry import TransformRegistry
registry = TransformRegistry(
owner="ACME Corporation",
author="John Doe <johndoe@acme.com>",
host_url="https://transforms.acme.org",
seed_ids=["demo"]
)
# The rest of these attributes are optional
# metadata
registry.version = "0.1"
# transform suffix to indicate datasource
registry.display_name_suffix = " [ACME]"
# reference OAuth settings
registry.oauth_settings_id = ['github-oauth']
# transforms/GreetPerson.py
...
from extensions import registry
@registry.register_transform(
display_name='Greet Person',
input_entity='maltego.Phrase',
description='Returns a phrase greeting a person on the graph.',
output_entities=['maltego.Phrase'],
disclaimer='This disclaimer is optional and has to be accepted before this transform is run'
)
class GreetPerson(DiscoverableTransform):
@classmethod
def create_entities(cls, request, response):
...
Pro Tip: If the display_name
is either None
or ""
, the registry will try to create a display name from the
class name:
DNSToIP
'DNS To IP'GreetPerson
'Greet Person'You can declare transform settings in a central location and add them to the registry.
These settings will apply to all transforms which can be very helpful for api keys.
# settings.py
from maltego_trx.decorator_registry import TransformSetting
api_key_setting = TransformSetting(name='api_key',
display_name='API Key',
setting_type='string',
global_setting=True)
# extensions.py
from settings import api_key_setting
from maltego_trx.decorator_registry import TransformRegistry
registry = TransformRegistry(
owner="ACME Corporation",
author="John Doe <johndoe@acme.com>",
host_url="https://transforms.acme.org",
seed_ids=["demo"]
)
registry.global_settings = [api_key_setting]
Settings that aren't required for every transform have to be added to the register_transform
decorator explicitly. To
access the setting on the request, use the id
property, which will have the global prefix if it's a global setting.
The name
property won't work on global settings.
# settings.py
...
language_setting = TransformSetting(name='language',
display_name="Language",
setting_type='string',
default_value='en',
optional=True,
popup=True)
# transforms/GreetPerson.py
...
from settings import language_setting
from maltego_trx.transform import DiscoverableTransform
@registry.register_transform(display_name="Greet Person",
input_entity="maltego.Phrase",
description='Returns a phrase greeting a person on the graph.',
settings=[language_setting])
class GreetPerson(DiscoverableTransform):
@classmethod
def create_entities(cls, request: MaltegoMsg, response: MaltegoTransform):
language = request.getTransformSetting(language_setting.id)
...
To export the configurations, use the registry methods write_transforms_config()
and write_settings_config()
. These
methods have to executed after they have been registered with the TRX server.
# project.py
import sys
import transforms
from maltego_trx.registry import register_transform_function, register_transform_classes
from maltego_trx.server import application
from maltego_trx.handler import handle_run
# register_transform_function(transform_func)
from extensions import registry
register_transform_classes(transforms)
registry.write_transforms_config()
registry.write_settings_config()
handle_run(__name__, sys.argv, application)
.mtz
config with your local TransformsSince maltego-trx>=1.6.0
you can generate an .mtz
config file with your local transforms.
If you're already using the TransformRegistry
, just invoke the write_local_config()
method.
# project.py
registry.write_local_mtz()
This will create a file called local.mtz
in the current directory. You can then import this file into Maltego and
start using your local transforms faster. Just remember that settings are not passed to local transforms.
The method takes in the same arguments as the interface in the Maltego client.
If you are using a virtualenv
environment, you might want to change the command
argument to use that.
# project.py
registry.write_local_mtz(
mtz_path: str = "./local.mtz", # path to the local .mtz file
working_dir: str = ".",
command: str = "python3", # for a venv you might want to use `./venv/bin/python3`
params: str = "project.py",
debug: bool = True
)
If you have old TRX transforms that are written as functions, they can be registered with the server using
the maltego_trx.registry.register_transform_function
method.
In order to port your old transforms, make two changes:
maltego_trx
package instead of from a local file.register_transform_function
in order for the transform to be registered in your project.For example
In the legacy transform file, change:
from Maltego import *
def old_transform(m):
To:
from maltego_trx.maltego import MaltegoTransform
def old_transform(m):
...
In the project.py
file add the following:
from maltego_trx.registry import register_transform_function
from legacy_transform import trx_DNS2IP
register_transform_function(trx_DNS2IP)
The following commands can be run using the project.py file.
python project.py runserver
Start a development server that you can use to develop new transforms.
python project.py list
List the available transforms together with their transform server URLs and local transform names.
The following constants can be imported from maltego_trx.maltego
.
Message Types:
UIM_FATAL
UIM_PARTIAL
UIM_INFORM
UIM_DEBUG
Please take note:
You need to enable the debug
filter option in the Desktop client Output window to view debug
transform messages.
Bookmark Colors:
BOOKMARK_COLOR_NONE
BOOKMARK_COLOR_BLUE
BOOKMARK_COLOR_GREEN
BOOKMARK_COLOR_YELLOW
BOOKMARK_COLOR_PURPLE
BOOKMARK_COLOR_RED
Link Styles:
LINK_STYLE_NORMAL
LINK_STYLE_DASHED
LINK_STYLE_DOTTED
LINK_STYLE_DASHDOT
Overlays:
Overlays Enums are imported from maltego_trx.overlays
Overlay OverlayPosition:
NORTH = "N"
SOUTH = "S"
WEST = "W"
NORTH_WEST = "NW"
SOUTH_WEST = "SW"
CENTER = "C"
Overlay Type
IMAGE = "image"
COLOUR = "colour"
TEXT = "text"
The request/maltego msg object given to the transform contains the information about the input entity.
Attributes:
Value: str
: The display value of the input entity on the graphWeight: int
: The weight of the input entitySlider: int
: Results slider setting in the clientType: str
: The input entity typeProperties: dict(str: str)
: A key-value dictionary of the input entity propertiesTransformSettings: dict(str: str)
: A key-value dictionary of the transform settingsGenealogy: list(dict(str: str))
: A key-value dictionary of the Entity genealogy, this is only applicable for
extended entities e.g. Website EntityMethods:
getProperty(name: str)
: Get a property value of the input entitygetTransformSetting(name: str)
: Get a transform setting valueclearLegacyProperties()
: Delete (duplicate) legacy properties from the input entity. This will not result in
property information being lost, it will simply clear out some properties that the TRX library duplicates on all
incoming Transform requests. In older versions of TRX, these Entity properties would have a different internal ID when
sent the server than what the Maltego client would advertise in the Entity Manager UI. For a list of Entities with
such properties and their corresponding legacy and actual IDs, see entity_property_map
in maltego_trx/entities.py
.
For the majority of projects this distinction can be safely ignored.Methods:
addEntity(type: str, value: str) -> Entity
: Add an entity to the transform response. Returns an Entity object
created by the method.addUIMessage(message: str, messageType='Inform')
: Return a UI message to the user. For message type, use a message
type constant.Methods:
setType(type: str)
: Set the entity type (e.g. "Phrase" for maltego.Phrase entity)setValue(value: str)
: Set the entity valuesetWeight(weight: int)
: Set the entity weightaddDisplayInformation(content: str, title: str)
: Add display information for the entity.addProperty(fieldName: str, displayName: str, matchingRule: str, value: str)
: Add a property to the entity. Matching
rule can be strict
or loose
.addOverlay(propertyName: str, position: OverlayPosition, overlay_type: OverlayType)
: Add an overlay to the entity.
OverlayPosition
and OverlayType
are defined in the maltego_trx.overlays
Overlay can be added as Text, Image or Color
person_name = request.Value
entity = response.addEntity(Phrase, "Hi %s, nice to meet you!" % person_name)
# Normally, when we create an overlay, we would reference a property name so that Maltego can then use the
# value of that property to create the overlay. Sometimes that means creating a dynamic property, but usually
# it's better to either use an existing property, or, if you created the Entity yourself, and only need the
# property for the overlay, to use a hidden property. Here's an example of using a dynamic property:
entity.addProperty(
'dynamic_overlay_icon_name',
displayName="Name for overlay image",
value="Champion" # references an icon in the Maltego client
)
entity.addOverlay('dynamic_overlay_icon_name', OverlayPosition.WEST, OverlayType.IMAGE)
# DISCOURAGED:
# You *can* also directly supply the string value of the property, however this is not recommended. Why? If
# the entity already has a property of the same ID (in this case, "DE"), then you would in fact be assigning the
# value of that property, not the string "DE", which is not the intention. Nevertheless, here's an example:
entity.addOverlay(
'DE', # name of an icon, however, could also accidentally be a property name
OverlayPosition.SOUTH_WEST,
OverlayType.IMAGE
)
# Overlays can also be used to display extra text on an entity:
entity.addProperty("exampleDynamicPropertyName", "Example Dynamic Property", "loose", "Maltego Overlay Testing")
entity.addOverlay('exampleDynamicPropertyName', OverlayPosition.NORTH, OverlayType.TEXT)
# Or a small color indicator:
entity.addOverlay('#45e06f', OverlayPosition.NORTH_WEST, OverlayType.COLOUR)
setIconURL(url: str)
: Set the entity icon URLsetBookmark(bookmark: int)
: Set bookmark color index (e.g. -1 for BOOKMARK_COLOR_NONE, 3 for BOOKMARK_COLOR_PURPLE)setNote(note: str)
: Set note contentsetGenealogy(genealogy: dict)
: Set genealogyLink Methods:
setLinkColor(color: str)
: Set the link color (e.g. hex "#0000FF" for blue)setLinkStyle(style: int)
: Set the link style index (e.g. 0 for LINK_STYLE_NORMAL, 2 for LINK_STYLE_DOTTED)setLinkThickness(thick: int)
: Set link thickness (default is 1)setLinkLabel(label: str)
: Set the label of the linkreverseLink()
: Reverse the link directionaddCustomLinkProperty(fieldName=None, displayName=None, value=None)
: Set a custom property for the link