Navigation

Compatibility Changes in MongoDB 4.2

Removal of MMAPv1 Storage Engine

MongoDB 4.2 removes support for the deprecated MMAPv1 storage engine.

If your 4.0 deployment uses MMAPv1, you must change the deployment to WiredTiger Storage Engine before upgrading to MongoDB 4.2. For details, see:

MMAPv1 Specific Configuration Options

MongoDB removes the following MMAPv1 specific configuration options:

Removed Configuration File Setting Removed Command-line Option
storage.mmapv1.journal.commitIntervalMs  
storage.mmapv1.journal.debugFlags mongod --journalOptions
storage.mmapv1.nsSize mongod --nssize
storage.mmapv1.preallocDataFiles mongod --noprealloc
storage.mmapv1.quota.enforced mongod --quota
storage.mmapv1.quota.maxFilesPerDB mongod --quotaFiles
storage.mmapv1.smallFiles mongod --smallfiles
storage.repairPath mongod --repairpath
replication.secondaryIndexPrefetch mongod --replIndexPrefetch

Note

Starting in version 4.2, MongoDB processes will not start with these options. Remove any MMAPv1 specific configuration options if using a WiredTiger deployment.

MMAPv1 Specific Parameters

MongoDB removes the following MMAPv1 parameters:

  • newCollectionsUsePowerOf2Sizes
  • replIndexPrefetch

MMAPv1 Specific Command

MongoDB removes the MMAPv1 specific touch command.

MMAPv1 Specific Options for Commands and Methods

MongoDB removes the MMAPv1 specific options:

MongoDB ignores the MMAPv1 specific option async for fsync.

Removed or Deprecated Commands and Methods

Remove Support for the group Command

Starting in version 4.2, MongoDB removes the group command (deprecated since version 3.4) and its mongo shell helper db.collection.group().

Use db.collection.aggregate() with the $group stage instead.

Remove Support for the eval Command

Starting in version 4.2, MongoDB removes the eval command. eval command has been deprecated since version 3.0.

The associated MongoDB 4.2 mongo shell methods db.eval() and db.collection.copyTo() can only be run when connected to MongoDB 4.0 or earlier.

Remove Support for the copydb and clone Commands

Starting in version 4.2, MongoDB removes the deprecated copydb command and clone command.

The corresponding mongo shell helpers db.copyDatabase() and db.cloneDatabase() can only be run when connected to MongoDB 4.0 or earlier.

As alternatives, users can use mongodump and mongorestore (with the mongorestore options --nsFrom and --nsTo) or write a script using the drivers.

For example, to copy the test database from a local instance running on the default port 27017 to the examples database on the same instance, you can:

  1. Use mongodump to dump the test database to an archive mongodump-test-db:

    mongodump --archive="mongodump-test-db" --db=test
    
  2. Use mongorestore with --nsFrom and --nsTo to restore (with database name change) from the archive:

    mongorestore --archive="mongodump-test-db" --nsFrom='test.*' --nsTo='examples.*'
    

Tip

Include additional options as necessary, such as to specify the uri or host, username, password and authentication database.

Alternatively, instead of using an archive file, you can mongodump the test database to the standard output stream and pipe into mongorestore:

mongodump --archive --db=test | mongorestore --archive  --nsFrom='test.*' --nsTo='examples.*'

Remove Support for the parallelCollectionScan Command

Starting in version 4.2, MongoDB removes the parallelCollectionScan command.

Remove maxScan

MongoDB removes the deprecated option maxScan for the find command and the mongo shell helper cursor.maxScan(). Use either the maxTimeMS option for the find command or the helper cursor.maxTimeMS() instead.

Remove Support for the geoNear Command

Starting in version 4.2, MongoDB removes the geoNear command. Use the $geoNear aggregation stage instead.

The options for $geoNear are similar to the removed geoNear command with the following exceptions:

  • The removed geoNear command includes in its output a field named dis that included the distance information.

    For the $geoNear stage, specify the distance field name in distanceField.

  • The removed geoNear command accepts a boolean value for the includeLocs option to include the loc field.

    For the $geoNear stage, specify the location field name in includeLocs.

  • The removed geoNear command includes the avgDistance and maxDistance of the returned results.

    You can use the aggregation pipeline to return the avgDistance and maxDistance as well. Specifically, after the $geoNear stage, include a $group stage to calculate the avgDistance and maxDistance:

    db.places.aggregate([
       { $geoNear: { near: <...>, distanceField: "dis", includeLocs: "loc", spherical: true, ... } },
       { $group: { _id: null, objectsLoaded: { $sum: 1 }, maxDistance:
             { $max: "$dis" }, avgDistance: { $avg: "$dis" } } }
    ])
    

Remove Support for the repairDatabase Command

Starting in version 4.2, MongoDB removes the repairDatabase command and its mongo shell helper db.repairDatabase() as well as the repairDatabase privilege.

As alternatives:

Remove Support for getPrevError Command

Starting in version 4.2, MongoDB removes the deprecated getPrevError command and its mongo shell helper db.getPrevError().

Deprecate Support for cloneCollection

MongoDB deprecates the cloneCollection command and its mongo shell helper db.cloneCollection()

As alternatives,

Deprecated Plan Cache Commands/Methods

MongoDB deprecates the following:

Aggregation

$out Stage Restrictions

$out and Views

The view definition pipeline cannot include the $out stage. If you already have an existing view that includes the $out stage, you can no longer create new views from this existing view.

For existing views that include the $out stage, you should either drop and recreate the views without the $out stage or use replace the view definition with a new pipeline that does not contain the $out stage.

$out and $lookup

The $lookup stage cannot include the $out stage in its nested pipeline field for the joined collection.

$out and linearizable Read Concern Level

The $out stage cannot be used in conjunction with read concern "linearizable".

$out and Explain

You cannot run the db.collection.explain() method (or the explain command) in executionStats mode or allPlansExecution mode if the aggregation pipeline contains the $out stage.

If the aggregation pipeline contains the $out stage, to view executionStats or allPlansExecution information, run explain without the $out stage in order to return explain results for the preceding stages.

Alternatively, you can run explain in queryPlanner mode for an aggregation pipeline that contains the $out stage.

$out and majority Read Concern Level

Starting in MongoDB 4.2, you can specify read concern level "majority" for an aggregation that includes an $out stage.

In MongoDB 4.0 and earlier, you cannot include the $out stage to use "majority" read concern for the aggregation.

Remove limit and num Options from $geoNear

Starting in version 4.2, MongoDB removes the limit and num options for the $geoNear stage as well as the default limit of 100 documents. To limit the results of $geoNear, use the $geoNear stage with the $limit stage.

For example, the following aggregation where the $geoNear stage contains the num option is no longer valid in 4.2.

db.places.aggregate([
   {
     $geoNear: {
        near: { type: "Point", coordinates: [ -73.99279 , 40.719296 ] },
        distanceField: "distance",
        num: 5,                 // Not supported in 4.2
        spherical: true
     }
   }
])

Instead, you can rewrite the aggregation to the following pipeline:

db.places.aggregate([
   {
     $geoNear: {
        near: { type: "Point", coordinates: [ -73.99279 , 40.719296 ] },
        distanceField: "distance",
        spherical: true
     }
   },
   { $limit: 5 }
])

Transactions

  • Starting in MongoDB 4.2, you cannot specify killCursors as the first operation in a transaction.
  • Starting in MongoDB 4.2, you cannot write to capped collections in transactions. Reads from capped collections are still supported in transactions.
  • Starting in MongoDB 4.2, MongoDB removes the 16MB total size limit for a transaction. In version 4.2, MongoDB creates as many oplog entries as necessary to the encapsulate all write operations in a transaction. In previous versions, MongoDB creates a single entry for all write operations in a transaction, thereby imposing a 16MB total size limit for a transaction.

Change Streams

Availability

Starting in MongoDB 4.2, change streams are available regardless of the "majority" read concern support; that is, read concern majority support can be either enabled (default) or disabled to use change streams.

In MongoDB 4.0 and earlier, change streams are available only if "majority" read concern support is enabled (default).

Default Collation

Starting in MongoDB 4.2, change streams use simple binary comparisons unless an explicit collation is provided. In earlier versions, change streams opened on a single collection (db.collection.watch()) would inherit that collection’s default collation.

Resume Token Modification

Starting in MongoDB 4.2, change streams will throw an exception if the change stream aggregation pipeline modifies an event’s _id field.

MongoDB Tools

FIPS Mode

Starting in version 4.2, MongoDB removes the --sslFIPSMode option for the following programs:

The programs will use FIPS compliant connections to mongod/mongos if the mongod/mongos instances are configured to use FIPS mode.

Extended JSON v2

Starting in version 4.2:

bsondump Uses Extended JSON v2.0 (Canonical mode) format.
mongodump

Use Extended JSON v2.0 (Canonical mode) format for the metadata. Requires mongorestore version 4.2+ that supports Extended JSON v2.0 (Canonical mode or Relaxed) format.

Tip

In general, use corresponding versions of mongodump and mongorestore. That is, to restore data files created with a specific version of mongodump, use the corresponding version of mongorestore.

mongoexport
Creates output data in Extended JSON v2.0 (Relaxed mode) by default.
Creates output data in Extended JSON v2.0 (Canonical mode) if used with --jsonFormat.
mongoimport
Expects import data to be in Extended JSON v2.0 (either Relaxed or Canonical mode) by default.
Can recognize data that is in Extended JSON v1.0 format if the option --legacy is specified.

Tip

In general, the versions of mongoexport and mongoimport should match. That is, to import data created from mongoexport, you should use the corresponding version of mongoimport.

For details on MongoDB extended JSON v2, see MongoDB Extended JSON (v2).

--query Options

Starting in version 4.2, the query option for mongodump --query and mongoexport --query must be in Extended JSON v2 format (relaxed or canonical/strict mode), including enclosing the field names and the operators in quotes, as in the following:

mongoexport -d=test -c=records -q='{ "a": { "$gte": 3 }, "date": { "$lt": { "$date": "2016-01-01T00:00:00.000Z" } } }' --out=exportdir/myRecords.json

In earlier versions, the query options uses the Extended JSON v1 format and the field names and the operators do not need to be in quotes:

mongoexport -d=test -c=records -q='{ a: { $gte: 3 }, date: { $lt: { "$date": "2016-01-01T00:00:00.000Z" } } }' --out=exportdir/myRecords.json

Replica Set State Changes

Primary Step Down

Starting in MongoDB 4.2, replSetStepDown (and replSetReconfig that results in a step down) no longer closes all client connections. However, writes that were in progress are killed.

In MongoDB 4.0 and earlier, replSetStepDown closes all client connections during the step down.

ROLLBACK State

Starting in version 4.2, MongoDB kills all in-progress user operations when a member enters the ROLLBACK state.

4.2 Drivers Enable Retryable Writes by Default

The official MongoDB 3.6 and 4.0-compatible drivers required including the retryWrites=true option in the connection string to enable retryable writes for that connection.

The official MongoDB 4.2-compatible drivers enable Retryable Writes by default. Applications upgrading to the 4.2-compatible drivers that require retryable writes may omit the retryWrites=true option. Applications upgrading to the 4.2-compatible drivers that require disabling retryable writes must include retryWrites=false in the connection string.

Important

The local database does not support retryable writes. Applications which write to the local database will encounter write errors upon upgrading to a 4.2-series driver unless retryable writes are explicitly disabled.

General Changes

Indexes

Stronger Restrictions on reIndex

MongoDB implements a stronger restriction on running reIndex command and db.collection.reIndex() shell helper against a collection in a sharded cluster by disallowing reIndex and db.collection.reIndex() on a mongos.

Restriction on db.collection.dropIndex() Method

You cannot specify db.collection.dropIndex("*") to drop all non-_id indexes. Use db.collection.dropIndexes() instead.

Duplicate Index Creation Attempt Error Message

MongoDB changes the returned response if you create an index with one name, and then try to create the index again with another name.

Starting in version 4.2, the createIndexes command and the mongo shell helpers db.collection.createIndex() and db.collection.createIndexes() report an error if you create an index with one name, and then try to create the same index again but with another name.

{
   "ok" : 0,
   "errmsg" : "Index with name: x_1 already exists with a different name",
   "code" : 85,
   "codeName" : "IndexOptionsConflict"
}

In previous versions, MongoDB did not create the index again, but would return a response object with ok value of 1 and a note that implied that the index was not recreated. For example:

{
   "numIndexesBefore" : 2,
   "numIndexesAfter" : 2,
   "note" : "all indexes already exist",
   "ok" : 1
}

Hashed Index on PowerPC

For hashed indexes, MongoDB 4.2 ensures that the hashed value for the floating point value 263 on PowerPC is consistent with other platforms. In previous versions, the hashed value for the floating point value 263 on PowerPC is inconsistent with other platforms.

Although hashed indexes on a field that may contain floating point values greater than 253 is an unsupported configuration, clients may still insert documents where the indexed field has the value 263.

To list all hashed indexes for your deployment, see PowerPC and 263.

If the current MongoDB 4.0 sharded cluster on PowerPC contains hashed values for 263 as part of the shard key, additional considerations must be taken before upgrading the sharded cluster to 4.2. See Upgrade a Sharded Cluster to 4.2.

min()/max()

Starting in MongoDB 4.2, when specifying min()/max() for a db.collection.find() operation, you must explicitly specify the index for min()/max() with the cursor.hint() method unless the find() query is an equality condition on the _id field { _id: <value> }.

Similarly, when specifying min/max in the find command, you must also explicitly specify the hint for the min/max index.

In previous versions, you could run min()/max() (or the corresponding min/max fields in the command) with or without explicitly hinting the index regardless of the query condition. If run without the hint in 4.0 and earlier, MongoDB selects the index using the fields in the indexBounds; however, if multiple indexes exist on same fields with different sort orders, the selection of the index may be ambiguous.

CurrentOp

  • When reporting on "getmore" operations, the $currentOp aggregation stage, along with currentOp command and the db.currentOp() helper, now returns the originatingCommand field as a nested field in the new cursor field. In previous versions, the originatingCommand was a top-level field for the associated "getmore" document. See also 4.2 currentOp Changes.

Server Status

Logging

  • When logging to syslog, the format of the message text includes the component. For example:

    ...  ACCESS   [repl writer worker 5] Unsupported modification to roles collection ...
    

    Previously, the syslog message text did not include the component. For example:

    ... [repl writer worker 1] Unsupported modification to roles collection ...
    
  • Starting in MongoDB 4.2, the getLog command truncates any event that contains more than 1024 characters. In earlier versions, getLog truncates after 512 characters.

  • Starting in version 4.2, MongoDB logs the debug verbosity level. For example, if verbosity level is 2, MongoDB logs D2.

    In previous versions, MongoDB log messages only specified D for Debug level.

Wire Protocol

  • MongoDB no longer supports the deprecated internal OP_COMMAND and the corresponding OP_COMMANDREPLY wire protocol.

killCursors Changes

Transactions

Starting in MongoDB 4.2, you cannot specify killCursors as the first operation in a transaction.

Privileges

Starting in MongoDB 4.2, users can always kill their own cursors, regardless of whether the users have the privilege to killCursors. As such, the killCursors privilege has no effect in MongoDB 4.2+.

In MongoDB 3.6.3 through MongoDB 4.0.x, users required the killCursors privilege in order to kill their own cursors when access control is enabled.

Removes AsyncRequestsSenderUseBaton Parameter

In MongoDB 4.2+ deployment, MongoDB removes the AsyncRequestsSenderUseBaton parameter and always enables the performance enhancement controlled by the parameter.

Stricter Validation of count Syntax

Starting in version 4.2, MongoDB implements a stricter validation of the option names for the count command. The command now errors if you specify an unknown option name.

In previous versions, MongoDB ignores invalid option names.

Causal Consistency Sessions

Starting in MongoDB 4.2, the following commands no longer support afterClusterTime:

As such, these operations cannot be associated with causally consistent sessions.

Removes fastmodinsert Metric

MongoDB 4.2 removes the deprecated fastmodinsert metric from various outputs, including the explain executionStats, the profiler output, etc.

Map-Reduce

Starting in version 4.2, MongoDB deprecates:

  • The map-reduce option to create a new sharded collection as well as the use of the sharded option for map-reduce. To output to a sharded collection, create the sharded collection first. MongoDB 4.2 also deprecates the replacement of an existing sharded collection.
  • The explicit specification of nonAtomic: false option.

Balancer State and Autosplit

Starting in MongoDB 4.2:

The mongo methods sh.enableBalancing(namespace) and sh.disableBalancing(namespace) have no affect on the auto-splitting.

Lock Diagnostics Reporting

Starting in version 4.2, MongoDB reports on ReplicationStateTransition lock information.

In addition, MongoDB 4.2 separates ParallelBatchWriterMode lock information from Global lock information. Earlier MongoDB versions report ParallelBatchWriterMode lock information as part of Global locks.

For operations that report on lock information, see:

findAndModify Query/Sort/Projection Argument Validation

Starting in MongoDB 4.2 (and 4.0.12+ and 3.6.14+), the findAndModify command and its associated mongo shell methods error if the specified query, sort, or projection argument is not a document.

In earlier versions, the operation treated non-document query or sort argument as an empty document {}.

See:

dropDatabase and movePrimary

Starting in MongoDB 4.2,

  • If you drop a database and create a new database with the same name, either:
  • If you use the movePrimary command to move unsharded collections, either:

This ensures that mongos and shard instances refresh their metadata cache. Otherwise, the you may miss data on reads, and may not write data to the correct shard. To recover, you must manually intervene.

In earlier versions, you only need to restart or run flushRouterConfig on the mongos instances.

For more information, see dropDatabase and movePrimary.

libldap and libldap_r

For MongoDB 4.2 (and 4.0.9) Enterprise binaries linked against libldap (such as when running on RHEL), access to the libldap is synchronized, incurring some performance/latency costs.

For MongoDB 4.2 (and 4.0.9) Enterprise binaries linked against libldap_r, there is no change in behavior from earlier MongoDB versions.

Connection Pooling and LDAP Server

Starting in version 4.2, MongoDB changes the ldapUseConnectionPool default values to:

  • true on Windows.
  • true on Linux where MongoDB Enterprise binaries are linked against libldap_r.

That is, on those systems, MongoDB, by default, uses connection pooling to connect to the LDAP server for authentication/authorization.

In earlier versions (versions 4.0.9+), MongoDB uses false as the default value for ldapUseConnectionPool. That is, MongoDB, by default, does not use connection pooling to connect to the LDAP server for authentication/authorization.

See ldapUseConnectionPool for details.

Removal of system.indexes and system.namespaces Collections

Starting in version 4.2, MongoDB removes the system.indexes and system.namespaces collections (deprecated since v3.0).

With the removal of these collections, built-in roles clusterManager, clusterMonitor, dbAdmin, read, restore, and other roles that inherit from these roles no longer provide privileges to directly access system.indexes and system.namespaces collections.

Arbiter Downgrade Requires Clearing Data Directory

MongoDB 4.2 arbiter data files are incompatible with MongoDB 4.0. Downgrading from MongoDB 4.2 to 4.0 requires deleting arbiter data files as an intermediary step. Running a MongoDB 4.0 arbiter against MongoDB 4.2 data files may result in unexpected behavior.

The downgrade instructions for replica sets and sharded clusters include specific steps for downgrading arbiters from 4.2 to 4.0:

Sharded Collections and Replace Documents

Starting in MongoDB 4.2,

  • Replace document operations, such as db.collection.replaceOne() or db.collection.update( query, replacement, … ), attempt to target a single shard, first by using the query filter. If the operation cannot target a single shard by the query filter, it then attempts to target by the replacement document.

    In earlier versions, the operation attempts to target using the replacement document.

  • For a replace document operation that includes upsert: true and is on a sharded collection, the filter must include an equality match on the full shard key.

4.2 Feature Compatibility

Some features in 4.2 require not just the 4.2 binaries but the featureCompatibilityVersion (fCV) set to 4.2. These features include:

  • Distributed transactions.
  • Removal of Index Key Limit for MongoDB versions with fCV set to 4.2+. In concert with the removal of this limit, the failIndexKeyTooLong parameter has no effect for MongoDB versions with fCV set to 4.2+ and only applies for MongoDB 2.6 through MongoDB versions with fCV set to "4.0" or earlier.
  • Removal of Index Name Length for MongoDB versions with fCV set to 4.2+.
  • New internal format for unique indexes. The new format applies to both existing unique indexes as well as newly created/rebuilt unique indexes.
  • Starting in MongoDB 4.2, users can no longer use the query filter $type: 0 as a synonym for $exists:false. To query for null or missing fields, see Query for Null or Missing Fields.
  • MongoDB 4.2 adds wildcard indexes to support workloads where users query against custom fields or a large variety of fields in a collection.