Quickstart

In the zeroless module, two classes can be used to define how distributed entities are related (i.e. Server and Client). To put it bluntly, with the exception of the pair pattern, a client may be connected to multiple servers, while a server may accept incoming connections from multiple clients.

Both servers and clients are able to create a callable and/or iterable, depending on the message passing pattern. So that you can iterate over incoming messages and/or call to transmit a message.

Message Passing Patterns

Zeroless supports the following message passing patterns:

Push-Pull

Useful for distributing the workload among a set of workers. A common pattern in the Stream Processing field, being the cornestone of applications like Apache Storm for instance. Also, it can be seen as a generalisation of the Map-Reduce pattern.

import logging

from zeroless import (Server, log)

# Setup console logging
consoleHandler = logging.StreamHandler()
log.setLevel(logging.DEBUG)
log.addHandler(consoleHandler)

# Binds the pull server to port 12345
# And assigns an iterable to wait for incoming messages
listen_for_push = Server(port=12345).pull()

for msg in listen_for_push:
    print(msg)

import logging

from zeroless import (Client, log)

# Setup console logging
consoleHandler = logging.StreamHandler()
log.setLevel(logging.DEBUG)
log.addHandler(consoleHandler)

# Connects the client to as many servers as desired
client = Client()
client.connect_local(port=12345)

# Initiate a push client
# And assigns a callable to push messages
push = client.push()

for msg in [b"Msg1", b"Msg2", b"Msg3"]:
    push(msg)

Publisher-Subscriber

Useful for broadcasting messages to a set of peers. A common pattern for allowing real-time notifications at the client side, without having to resort to inneficient approaches like pooling. Online services like PubNub or IoT protocols like MQTT are examples of this pattern usage.

import logging

from zeroless import (Client, log)

# Setup console logging
consoleHandler = logging.StreamHandler()
log.setLevel(logging.DEBUG)
log.addHandler(consoleHandler)

# Connects the client to as many servers as desired
client = Client()
client.connect_local(port=12345)

# Initiate a subscriber client
# Assigns an iterable to wait for incoming messages with the topic 'sh'
listen_for_pub = client.sub(topics=[b'sh'])

for topic, msg in listen_for_pub:
    print(topic, ' - ', msg)

import logging

from time import sleep

from zeroless import (Server, log)

# Setup console logging
consoleHandler = logging.StreamHandler()
log.setLevel(logging.DEBUG)
log.addHandler(consoleHandler)

# Binds the publisher server to port 12345
# And assigns a callable to publish messages with the topic 'sh'
pub = Server(port=12345).pub(topic=b'sh', embed_topic=True)

# Gives publisher some time to get initial subscriptions
sleep(1)

for msg in [b"Msg1", b"Msg2", b"Msg3"]:
    pub(msg)

Note: ZMQ’s topic filtering capabilities are publisher side since ZMQ 3.0.

Last but not least, SUB sockets that bind will not get any message before they first ask for via the provided generator, so prefer to bind PUB sockets if missing some messages is not an option.

Request-Reply

Useful for RPC style calls. A common pattern for clients to request data and receive a response associated with the request. The HTTP protocol is well-known for adopting this pattern, being it essential for Restful services.

import logging

from zeroless import (Server, log)

# Setup console logging
consoleHandler = logging.StreamHandler()
log.setLevel(logging.DEBUG)
log.addHandler(consoleHandler)

# Binds the reply server to port 12345
# And assigns a callable and an iterable
# To both transmit and wait for incoming messages
reply, listen_for_request = Server(port=12345).reply()

for msg in listen_for_request:
    print(msg)
    reply(msg)

import logging

from zeroless import (Client, log)

# Setup console logging
consoleHandler = logging.StreamHandler()
log.setLevel(logging.DEBUG)
log.addHandler(consoleHandler)

# Connects the client to as many servers as desired
client = Client()
client.connect_local(port=12345)

# Initiate a request client
# And assigns a callable and an iterable
# To both transmit and wait for incoming messages
request, listen_for_reply = client.request()

for msg in [b"Msg1", b"Msg2", b"Msg3"]:
    request(msg)
    response = next(listen_for_reply)
    print(response)

Pair

More often than not, this pattern will be unnecessary, as the above ones or the mix of them suffices most use cases in distributed computing. Regarding its capabilities, this pattern is the most similar alternative to usual posix sockets among the aforementioned patterns. Therefore, expect one-to-one and bidirectional communication.

import logging

from zeroless import (Server, log)

# Setup console logging
consoleHandler = logging.StreamHandler()
log.setLevel(logging.DEBUG)
log.addHandler(consoleHandler)

# Binds the pair server to port 12345
# And assigns a callable and an iterable
# To both transmit and wait for incoming messages
pair, listen_for_pair = Server(port=12345).pair()

for msg in listen_for_pair:
    print(msg)
    pair(msg)

import logging

from zeroless import (Client, log)

# Setup console logging
consoleHandler = logging.StreamHandler()
log.setLevel(logging.DEBUG)
log.addHandler(consoleHandler)

# Connects the client to a single server
client = Client()
client.connect_local(port=12345)

# Initiate a pair client
# And assigns a callable and an iterable
# To both transmit and wait for incoming messages
pair, listen_for_pair = client.pair()

for msg in [b"Msg1", b"Msg2", b"Msg3"]:
    pair(msg)
    response = next(listen_for_pair)
    print(response)

Additional Features

Logging

Python provides a wonderfull logging module. It can be used to track Zeroless’ internal workflow in a modular way, therefore being very useful for debugging purposes.

The zeroless module allows logging via a global Logger object.

from zeroless import log

To enable it, just add an Handler object and set an appropriate logging level.

Multipart Messages

In the Zeroless API, all callables have a print like signature, therefore being able to have an infinite number of arguments. Each of these arguments are part of the whole message, that could be divided in multiple pieces. Being that useful when you have a simple message structure, with just a few fields, and don’t want to rely on a data formatting standard (e.g. JSoN, XML) to maintain the message semantics. Also, given the need to parse those different parts that a single message may have, the receiver’s iterable will return them all, at once, in transparent fashion.

For more on this, see the examples/multipart folder or check the following example:

from zeroless import Server

# Binds the pull server to port 12345
# And assigns an iterable to wait for incoming messages
listen_for_push = Server(port=12345).pull()

for id, msg in listen_for_push:
    print(id, ' - ', msg)

from zeroless import Client

# Connects the client to as many servers as desired
client = Client()
client.connect_local(port=12345)

# Initiate a push client
# And assigns a callable to push messages
push = client.push()

for id, msg in [(b"1", b"Msg1"), (b"2", b"Msg2"), (b"3", b"Msg3")]:
    push(id, msg)