Welcome to jaraco.mongodb documentation!#

History

Decode support for JSON strings using ordered dictionaries and parsing MongoDB-specific objects like dates.

For example, if you have a JSON object representing a sort, you want to retain the order of keys:

>>> ob = decode('{"key1": 1, "key2": 2}')
>>> list(ob.keys())
['key1', 'key2']

Or if you want to query by a date, PyMongo needs a Python datetime object, which has no JSON representation, so this codec converts $date keys to date objects.

>>> ob = decode('{"$gte": {"$date": "2019-01-01"}}')
>>> ob['$gte']
datetime.datetime(2019, 1, 1, 0, 0)

This function is useful in particular if you’re accepting JSON queries over an HTTP connection and you don’t have the luxury of Javascript expressions like you see in the Mongo shell or Compass.

jaraco.mongodb.codec.maybe_date(obj)#

>>> maybe_date({"$date": "2019-01-01"})
datetime.datetime(2019, 1, 1, 0, 0)

Backslash-escape text such that it is safe for MongoDB fields, honoring Restrictions on Field Names.

>>> decode(encode('my text with dots...'))
'my text with dots...'

>>> decode(encode(r'my text with both \. and literal .'))
'my text with both \\. and literal .'

>>> decode(encode('$leading dollar'))
'$leading dollar'

jaraco.mongodb.fields.decode(encoded)#

jaraco.mongodb.fields.encode(text)#

jaraco.mongodb.fields.unescape(match)#

Helper functions to augment PyMongo

jaraco.mongodb.helper.connect_db(uri, default_db_name=None, factory=pymongo.MongoClient)#

Use pymongo to parse a uri (possibly including database name) into a connected database object.

This serves as a convenience function for the common use case where one wishes to get the Database object and is less concerned about the intermediate MongoClient object that pymongo creates (though the connection is always available as db.client).

>>> db = connect_db(
...     'mongodb://mongodb.localhost/mydb?readPreference=secondary')
>>> db.name
'mydb'
>>> db.client.read_preference
Secondary(...)

If no database is indicated in the uri, fall back to default.

>>> db = connect_db('mongodb://mgo/', 'defaultdb')
>>> db.name
'defaultdb'

The default should only apply if no db was present in the URI.

>>> db = connect_db('mongodb://mgo/mydb', 'defaultdb')
>>> db.name
'mydb'

jaraco.mongodb.helper.connect_gridfs(uri, db=None)#: Construct a GridFS instance for a MongoDB URI.

jaraco.mongodb.helper.get_collection(uri)#

jaraco.mongodb.helper.server_version(conn)#

>>> conn = getfixture('mongodb_instance').get_connection()
>>> ver = server_version(conn)
>>> len(ver)
3
>>> set(map(type, ver))
{<class 'int'>}

Migration support as features in MongoWorld 2016 From the Polls to the Trolls.

The Manager class provides a general purpose support for migrating documents to a target version through a series of migration functions.

class jaraco.mongodb.migration.Manager(target_version)#

Bases: object

A manager for facilitating the registration of migration functions and applying those migrations to documents.

To use, implement migration functions to and from each adjacent version of your schema and decorate each with the register classmethod. For example:

>>> @Manager.register
... def v1_to_2(manager, doc):
...     doc['foo'] = 'bar'
>>> @Manager.register
... def v2_to_3(manager, doc):
...     doc['foo'] = doc['foo'] + ' baz'

Note that in addition to the document, the migration manager is also passed to the migration function, allowing for other context to be made available during the migration.

To create a manager for migrating documents to version 3:

>>> mgr = Manager(3)

Then, use the manager to migrate a document to a target version.

>>> v1_doc = dict(version=1, data='stub')
>>> v3_doc = mgr.migrate_doc(v1_doc)

>>> v3_doc['version']
3
>>> v3_doc['foo']
'bar baz'

Note that the document is modified in place:

>>> v1_doc is v3_doc
True

>>> Manager._upgrade_funcs.clear()

migrate_doc(doc)#: Migrate the doc from its current version to the target version and return it.

classmethod register(func)#: Decorate a migration function with this method to make it available for migrating cases.

version_attribute_name = 'version'#

class jaraco.mongodb.oplog.NiceRepr(orig)#

Bases: object

Adapt a Python representation of a MongoDB object to make it appear nicely when rendered as a string.

>>> messy_doc = collections.OrderedDict([
...     ('ts', bson.Timestamp(1111111111, 30),
... )])
>>> print(NiceRepr(messy_doc))
{"ts": {"$timestamp": {"t": 1111111111, "i": 30}}}

class jaraco.mongodb.oplog.NullResumeFile#

Bases: object

read()#

save(ts)#

class jaraco.mongodb.oplog.Oplog(coll)#

Bases: object

find_params: Dict[str, Any] = {}#

get_latest_ts()#

has_ops_before(ts)#: Determine if there are any ops before ts

query(spec)#

since(ts)#: Query the oplog for items since ts and then return

class jaraco.mongodb.oplog.RenameSpec(old_ns, new_ns)#

Bases: object

affects(ns)#

classmethod from_spec(string_spec)#: Construct RenameSpec from a pair separated by equal sign (‘=’).

class jaraco.mongodb.oplog.Renamer(iterable=(), /)#

Bases: list

>>> specs = [
...      'a=b',
...      'alpha=gamma',
...  ]
>>> renames = Renamer.from_specs(specs)
>>> op = dict(ns='a.a')
>>> renames(op)
>>> op['ns']
'b.a'
>>> renames.affects('alpha.foo')
True
>>> renames.affects('b.gamma')
False

affects(ns)#: Return True if this renamer affects the indicated namespace.

classmethod from_specs(specs)#

invoke(op)#: Replace namespaces in op based on RenameSpecs in self.

class jaraco.mongodb.oplog.ResumeFile#

Bases: str

read()#: Read timestamp from file.

save(ts)#: Save timestamp to file.

class jaraco.mongodb.oplog.TailingOplog(coll)#

Bases: Oplog

find_params: Dict[str, Any] = {'cursor_type': 34, 'oplog_replay': True}#

since(ts)#: Tail the oplog, starting from ts.

class jaraco.mongodb.oplog.Timestamp(time: datetime | int, inc: int)#

Bases: Timestamp

dump(stream)#

Serialize self to text stream.

Matches convention of mongooplog.

classmethod for_window(window)#: Given a timedelta window, return a timestamp representing that time.

classmethod load(stream)#

Load a serialized version of self from text stream.

Expects the format used by mongooplog.

classmethod wrap(orig)#: Wrap an original timestamp as returned by a pymongo query with a version of this class.

jaraco.mongodb.oplog.applies_to_ns(op, ns)#

jaraco.mongodb.oplog.apply(db, op)#: Apply operation in db

jaraco.mongodb.oplog.delta_from_seconds(seconds)#

jaraco.mongodb.oplog.main()#

jaraco.mongodb.oplog.parse_args(*args, **kwargs)#

Parse the args for the command.

It should be possible for one to specify ‘–ns’, ‘-x’, and ‘–rename’ multiple times:

>>> args = parse_args(['--ns', 'foo', 'bar', '--ns', 'baz'])
>>> args.ns
['foo', 'bar', 'baz']

>>> parse_args(['-x', '--exclude']).exclude
[]

>>> renames = parse_args(['--rename', 'a=b', '--rename', 'b=c']).rename
>>> len(renames)
2

“…” below should be “jaraco.” but for pytest-dev/pytest#3396 >>> type(renames) <class ‘…mongodb.oplog.Renamer’>

jaraco.mongodb.oplog.string_none(value)#: Convert the string ‘none’ to None

jaraco.mongodb.query.compat_explain(cur)#: Simulate MongoDB 3.0 explain result on prior versions. http://docs.mongodb.org/v3.0/reference/explain-results/

jaraco.mongodb.query.project(*args, **kwargs)#

Build a projection for MongoDB.

Due to https://jira.mongodb.org/browse/SERVER-3156, until MongoDB 2.6, the values must be integers and not boolean.

>>> project(a=True) == {'a': 1}
True

Once MongoDB 2.6 is released, replace use of this function with a simple dict.

jaraco.mongodb.query.upsert_and_fetch(coll, doc, **kwargs)#

Fetch exactly one matching document or upsert the document if not found, returning the matching or upserted document.

See https://jira.mongodb.org/browse/SERVER-28434 describing the condition where MongoDB is uninterested in providing an upsert and fetch behavior.

>>> instance = getfixture('mongodb_instance').get_connection()
>>> coll = instance.test_upsert_and_fetch.items
>>> doc = {'foo': 'bar'}
>>> inserted = upsert_and_fetch(coll, doc)
>>> inserted
{...'foo': 'bar'...}
>>> upsert_and_fetch(coll, doc) == inserted
True

A MongoDB-backed CherryPy session store.

Although this module requires CherryPy, it does not impose the requirement on the package, as any user of this module will already require CherryPy.

To enable these sessions, your code must call Session.install() and then configure the CherryPy endpoint to use MongoDB sessions. For example:

jaraco.mongodb.sessions.Session.install()

session_config = {
    'sessions.on': True,
    'sessions.storage_type': 'MongoDB',
    'sessions.database': pymongo.MongoClient().database,
}
config = {
    '/': session_config,
}

cherrypy.quickstart(..., config=config)

The jaraco.modb module implements the codec interface, so may be used to encode more complex objects in the session:

session_config.update({
    'sessions.codec': jaraco.modb,
})

exception jaraco.mongodb.sessions.LockTimeout#: Bases: RuntimeError

class jaraco.mongodb.sessions.NullCodec#

Bases: object

decode(data)#

encode(data)#

class jaraco.mongodb.sessions.Session(id, **kwargs)#

Bases: Session

A MongoDB-backed CherryPy session store. Takes the following params:

database: the pymongo Database object.

collection_name: The name of the collection to use in the db.

codec: An object with ‘encode’ and ‘decode’ methods, used to encode objects before saving them to MongoDB and decode them when loaded from MongoDB.

lock_timeout: A timedelta or numeric seconds indicating how long to block acquiring a lock. If None (default), acquiring a lock will block indefinitely.

acquire_lock()#: Acquire the lock. Blocks indefinitely until lock is available unless lock_timeout was supplied. If the lock_timeout elapses, raises LockTimeout.

codec = <jaraco.mongodb.sessions.NullCodec object>#: by default, objects are passed directly to MongoDB

property collection#

classmethod install()#: Add this session to the cherrypy session handlers. CherryPy looks for session classes in vars(cherrypy.lib.sessions) with a name in title-case followed by “Session”.

release_lock()#

setup_expiration()#: Use pymongo TTL index to automatically expire sessions.

jaraco.mongodb.sharding.create_db_in_shard(db_name, shard, client=None)#: In a sharded cluster, create a database in a particular shard.

jaraco.mongodb.sharding.get_ids(collection)#

class jaraco.mongodb.timers.NeverExpires#

Bases: object

expired()#

class jaraco.mongodb.timers.Timer(expiration)#

Bases: object

A simple timer that will indicate when an expiration time has passed.

classmethod after(elapsed)#: Return a timer that will expire after elapsed passes.

expired()#

jaraco.mongodb.uri.join(base, new)#: Use urllib.parse to join the MongoDB URIs. Registers the MongoDB scheme first.

Welcome to jaraco.mongodb documentation!#

Indices and tables#