Reactive Scala Driver for MongoDB

Asynchronous & Non-Blocking

Release details

This is the release details for ReactiveMongo 1.1.0-RC12.

The documentation is available online, and its code samples are compiled to make sure it’s up-to-date.

Slides for these release notes are available.

What’s new?

Compatibility

This release is compatible with the following runtime.

MongoDB versions older than 3.0 are not longer (end of life 2018-2).

Recommended configuration:

The driver core and the modules are tested in a container based environment, with the specifications as bellow.

This can be considered as a recommended environment.

Migration

A Scalafix module is available to migrate from ReactiveMongo 0.12+ to 1.0.

To apply the migration rules, first setup Scalafix in the SBT build, then configure the ReactiveMongo rules as bellow.

scalafixDependencies in ThisBuild ++= Seq(
  "org.reactivemongo" %% "reactivemongo-scalafix" % "1.1.0-RC12")

Once the rules are configured, they can be applied from SBT.

scalafix ReactiveMongoUpgrade
scalafix ReactiveMongoLinter --check

Then upgrade the appropriate libraryDependencies in the SBT build, and re-recompile it.

sbt clean compile

Finally, apply manually the remaining fixes due to the breaking changes.

Suggest an improvement to these rules

Connection

The MongoDriver type is replaced by AsyncDriver, with asynchronous methods.

The utility function MongoConnection.parseURI is replaced by asynchronous function .fromString.

Also, the following options are deprecated:

SNI is now supported for the SSL connection.

The x.509 certificate authentication is now supported, and can be configured by setting x509 as authenticationMechanism, and with the following new options.

import reactivemongo.api._

def connection(driver: AsyncDriver) = driver.connect(
  "mongodb://server:27017/db?ssl=true"
  + "&authenticationMechanism=x509"
  + "&keyStore=file:///path/to/keystore.p12"
  + "&keyStoreType=PKCS12")

The DNS seedlist is now supported, using mongodb+srv:// scheme in the connection URI. It’s also possible to specify the credential directly in the URI.

import reactivemongo.api._

def seedListCon(driver: AsyncDriver) = driver.connect(
  "mongodb+srv://usr:pass@mymongo.mydomain.tld/mydb")

See documentation

Netty native

The internal Netty dependency has been updated to the version 4.1.

It comes with various improvements (memory consumption, …), and also to use Netty native support (kqueue for MacOS X and epoll for Linux, on x86_64 arch).

Note: The Netty dependency is shaded so it won’t conflict with any Netty version in your environment.

See documentation

BSON library

The Biːsən is the new default BSON library, that fixes some issues (OOM), and brings multiple API and performance improvements (simpler & better).

Highlights:

Documents and values

The API for BSONDocument has been slightly updated, with the function getAs renamed as getAsOpt (to be consistent with getAsTry).

New getOrElse function is also added.

import reactivemongo.api.bson._

def withFallback(doc: BSONDocument): String = {
  doc.getOrElse[String]("NAME", "defaultValue")
  // Equivalent to:
  //   doc.getAsOpt[String]("NAME").getOrElse("defaultValue")
}

New field utilities are provided for the most common types:

import reactivemongo.api.bson._

def foo(doc: BSONDocument): Unit = {
  val i: Option[Int] = doc.int("fieldNameInt")
  val d: Option[Double] = doc.double("fieldNameDouble")
  val l: Option[Long] = doc.long("fieldNameLong")
  val s: Option[String] = doc.string("fieldNameStr")
  val a: Option[Seq[BSONValue]] = doc.array("fieldNameArr")
  val b: Option[Boolean] = doc.booleanLike("fieldNameBool")
  val c: Option[BSONDocument] = doc.child("nestedDoc")
  val _: List[BSONDocument] = doc.children("arrayOfDocs")
}

The Biːsən library supports BSON Decimal128 (MongoDB 3.4+).

See documentation

Note: The BSONDocument and BSONArray factories have been optimized and support more use cases.

Coefficient between new/old throughput (op/s; =1: no change, 1+: better thrpt)

Source: BSONDocumentBenchmark, BSONDocumentHandlerBenchmark

Coefficient between new/old throughput (op/s; =1: no change, 1+: better thrpt)

Source: BSONArrayBenchmark

Reader and writer typeclasses

The names of these typeclasses are unchanged (BSONReader and BSONWriter), except the package that is reactivemongo.api.bson (instead of reactivemongo.bson).

In the previous BSON library, BSONReader and BSONWriter are defined with two type parameters:

BSONReader[B <: BSONValue, T]

BSONWriter[T, B <: BSONValue]

The new API has been simplified, with only the T type parameter kept.

import reactivemongo.api.bson._

// Read a String from BSON,
// whatever is the specific BSON value type
def stringReader: BSONReader[String] = ???

Not only it makes the API simpler, but it also allows to read different BSON types as a target Scala type(before it only supported for numeric/boolean, using the dedicated typeclasses).

For example, the Scala numeric types (BigDecimal, Double, Float, Int, Long) can be directly read from any consistent BSON numeric type (e.g. 1.0 as integer 1), without having to use BSONNumberLike.

import reactivemongo.api.bson._

BSONDouble(1.0D).asTry[Int]
// => Success(1)

Also, handler functions readTry and writeTry returns Try, for a safer representation of possible failures.

The new API is also safer, replacing BSONReader.read and BSONWriter.write respectively with BSONReader.readTry and BSONWriter.writeTry, so that serialization errors can be handle at typelevel.

In a similar way, BSONObjectID.parse now returns Try, and BSONValue.as is replaced by asOpt and asTry.

Like the current BSON library, some specific typeclasses are available (with same names) to only work with BSON documents: BSONDocumentReader and BSONDocumentWriter.

Some new handlers are provided by default, like those for Java Time types.

Note: The handler for java.util.Date is replaced the handler for java.time.Instant.

The error handling has also been improved, with more details.

Note: DocumentKeyNotFoundException in the previous API is replaced with BSONValueNotFoundException in the new one.

Coefficient between new/old throughput (op/s; =1: no change, 1+: better thrpt)

Source: BSON reader benchmarks

Map handler

A handler is now available to write and read Scala Map as BSON, provided the value types are supported.

import scala.util.Try
import reactivemongo.api.bson._

def bsonMap = {
  val input: Map[String, Int] = Map("a" -> 1, "b" -> 2)

  // Ok as key and value (String, Int) are provided BSON handlers
  val doc: Try[BSONDocument] = BSON.writeDocument(input)

  val output = doc.flatMap { BSON.readDocument[Map[String, Int]](_) }
}

You can to serialize a Map whose key type is not String, using the new typeclasses KeyWriter and KeyReader.

import scala.util.Try
import reactivemongo.api.bson._

final class FooKey(val value: String)

object FooKey {
  val bar = new FooKey("bar")
  val lorem = new FooKey("lorem")

  implicit val kw: KeyWriter[FooKey] = KeyWriter[FooKey](_.value)
  implicit val kr: KeyReader[FooKey] = KeyReader[FooKey] { new FooKey(_) }
}

def bsonMapCustomKey = {
  val input: Map[FooKey, Int] = Map(FooKey.bar -> 1, FooKey.lorem -> 2)

  // Ok as key and value (String, Int) are provided handlers
  val doc: Try[BSONDocument] = BSON.writeDocument(input)

  val output = doc.flatMap { BSON.readDocument[Map[FooKey, Int]](_) }
}
Iterable factories

New factories to handle BSON array are provided.

Considering the Element type as below:

import reactivemongo.api.bson.Macros

case class Element(str: String, v: Int)

val elementHandler = Macros.handler[Element]

The { BSONReader, BSONWriter }.sequence factories can be used to represent sequence of Element in BSON.

import reactivemongo.api.bson.BSONWriter

val seqWriter: BSONWriter[Seq[Element]] =
  BSONWriter.sequence[Element](elementHandler writeTry _)

// ---

seqWriter.writeTry(Seq(Element("foo", 1), Element("bar", 2)))
// Success: [ { 'str': 'foo', 'v': 1 }, { 'str': 'bar', 'v': 2 } ]

The { BSONReader, BSONWriter }.iterable factories can also be used (for example to work with Set).

import reactivemongo.api.bson.{ BSONArray, BSONDocument, BSONReader }

val setReader: BSONReader[Set[Element]] =
  BSONReader.iterable[Element, Set](elementHandler readTry _)

// ---

val itFixture2 = BSONArray(
  BSONDocument("str" -> "foo", "v" -> 1),
  BSONDocument("str" -> "bar", "v" -> 2))

setReader.readTry(itFixture2)
// Success: Set(Element("foo", 1), Element("bar", 2))
Tuple factories

New factories to create handler for tuple types (up to 5-arity) are provided.

If an array is the wanted BSON representation:

import reactivemongo.api.bson.{ BSONArray, BSONReader, BSONWriter }

val readerArrAsStrInt = BSONReader.tuple2[String, Int]
val writerStrIntToArr = BSONWriter.tuple2[String, Int]

val arr = BSONArray("Foo", 20)

readerArrAsStrInt.readTry(arr) // => Success(("Foo", 20))

writerStrIntToArr.writeTry("Foo" -> 20)
// => Success: arr = ['Foo', 20]

If a document representation is wanted:

import reactivemongo.api.bson.{
  BSONDocument, BSONDocumentReader, BSONDocumentWriter
}

val writerStrIntToDoc =
  BSONDocumentWriter.tuple2[String, Int]("name", "age")

writerStrIntToDoc.writeTry("Foo" -> 20)
// => Success: {'name': 'Foo', 'age': 20}

val readerDocAsStrInt =
  BSONDocumentReader.tuple2[String, Int]("name", "age")

readerDocAsStrInt.readTry(BSONDocument("name" -> "Foo", "age" -> 20))
// => Success(("Foo", 20))
Partial function

There are new factories based on partial functions: collect and collectFrom.

BSON reader:

import reactivemongo.api.bson.{ BSONReader, BSONInteger }

val intToStrCodeReader = BSONReader.collect[String] {
  case BSONInteger(0) => "zero"
  case BSONInteger(1) => "one"
}

intToStrCodeReader.readTry(BSONInteger(0)) // Success("zero")

intToStrCodeReader.readTry(BSONInteger(2))
// => Failure(ValueDoesNotMatchException(..))

BSON writer:

import reactivemongo.api.bson.{ BSONWriter, BSONInteger }

val strCodeToIntWriter = BSONWriter.collect[String] {
  case "zero" => BSONInteger(0)
  case "one" => BSONInteger(1)
}

strCodeToIntWriter.writeTry("zero") // Success(BSONInteger(0))

strCodeToIntWriter.writeTry("3")
// => Failure(IllegalArgumentException(..))

BSON document writer:

import reactivemongo.api.bson.{ BSONDocument, BSONDocumentWriter }

case class Bar(value: String)

val writer2 = BSONDocumentWriter.collectFrom[Bar] {
  case Bar(value) if value.nonEmpty =>
    scala.util.Success(BSONDocument("value" -> value))
}

Macros

The new library also provides similar macros, to materialized document readers and writers for case classes and sealed traits.

case class Person(name: String, age: Int)

import reactivemongo.api.bson._

val personHandler: BSONDocumentHandler[Person] =
  Macros.handler[Person]

// Or only ...
val personReader: BSONDocumentReader[Person] =
  Macros.reader[Person]

val personWriter: BSONDocumentWriter[Person] =
  Macros.writer[Person]

There is a new configuration mechanism, to specify a field naming and customize the name of each BSON field corresponding to Scala.

import reactivemongo.api.bson._

val withPascalCase: BSONDocumentHandler[Person] = {
  implicit def cfg: MacroConfiguration = MacroConfiguration(
    fieldNaming = FieldNaming.PascalCase)

  Macros.handler[Person]
}

withPascalCase.writeTry(Person(name = "Jane", age = 32))
// Success { "Name": "Jane", "Age": 32) }

With sealed family/trait, it’s also possible to configure the discriminator field and discriminator values according Scala types.

import reactivemongo.api.bson._

sealed trait Family1
case class Foo1(bar: String) extends Family1
case class Lorem1(ipsum: Int) extends Family1

implicit val foo1Handler = Macros.handler[Foo1]
implicit val lorem1Handler = Macros.handler[Lorem1]

val family1Handler: BSONDocumentHandler[Family1] = {
  implicit val cfg: MacroConfiguration = MacroConfiguration(
    discriminator = "_type",
    typeNaming = TypeNaming.SimpleName.andThen(_.toLowerCase))

  Macros.handler[Family1]
}

The nested type Macros.Options is replaced by similar type MacrosOptions.

Note: The Macros.Options.SaveSimpleName of the previous BSON library has been removed in favour of a configuration factory using similar TypeNaming.

The compile-time option AutomaticMaterialization has been added, when using the macros with sealed family, to indicate when you want to automatically materialize instances for the subtypes (if missing from the implicit scope).

sealed trait Color

case object Red extends Color
case object Blue extends Color
case class Green(brightness: Int) extends Color
case class CustomColor(code: String) extends Color

object Color {
  import reactivemongo.api.bson.{ Macros, MacroOptions },
    MacroOptions.{ AutomaticMaterialization, UnionType, \/ }

  // Use `UnionType` to define a subset of the `Color` type,
  type PredefinedColor =
    UnionType[Red.type \/ Green \/ Blue.type]
    with AutomaticMaterialization

  val predefinedColor = Macros.handlerOpts[Color, PredefinedColor]
}

Using the new option ReadDefaultValues, the default values can be used by readers when there is no BSON value.

import reactivemongo.api.bson.{
  BSONDocument, BSONDocumentReader, Macros, MacroOptions
}

case class FooWithDefault1(id: Int, title: String = "default")

{
  val reader: BSONDocumentReader[FooWithDefault1] =
    Macros.using[MacroOptions.ReadDefaultValues].
      reader[FooWithDefault1]

  reader.readTry(BSONDocument("id" -> 1)) // missing BSON title
  // => Success: FooWithDefault1(id = 1, title = "default")
}

Note: A new option DisableWarnings allows to specifically exclude macro warnings.

New macros for Value classes are new available.

package object relexamples {
  import reactivemongo.api.bson.{
    BSONHandler, BSONReader, BSONWriter, Macros
  }

  final class FooVal(val value: String) extends AnyVal

  val vh: BSONHandler[FooVal] = Macros.valueHandler[FooVal]
  val vr: BSONReader[FooVal] = Macros.valueReader[FooVal]
  val vw: BSONWriter[FooVal] = Macros.valueWriter[FooVal]
}
Annotations

The way Option is handled by the macros has been improved, also with a new annotation @NoneAsNull, which write None values as BSONNull (instead of omitting field/value).

import reactivemongo.api.bson.BSONDocument
import reactivemongo.api.bson.Macros, Macros.Annotations.NoneAsNull

case class WithNull(
  name: String,
  @NoneAsNull score: Option[Int])

Macros.writer[WithNull].writeTry(WithNull("foo", None))

// Serialized with null: {'name': 'foo', 'score': null}
// Rather than: {'name': 'foo'}

Also, a new annotation @Flatten has been added, to indicate to the macros that the representation of a property must be flatten rather than a nested document.

import reactivemongo.api.bson.BSONDocument
import reactivemongo.api.bson.Macros.Annotations.Flatten

case class Range(start: Int, end: Int)

case class LabelledRange(
  name: String,
  @Flatten range: Range)

// Flattened with macro as bellow:
BSONDocument("name" -> "foo", "start" -> 0, "end" -> 1)

// Rather than:
// BSONDocument("name" -> "foo", "range" -> BSONDocument(
//   "start" -> 0, "end" -> 1))

The new @DefaultValue can be used with ReadDefaultValues to specify a default value only used when reading from BSON.

import reactivemongo.api.bson.{
  BSONDocument, BSONDocumentReader, Macros, MacroOptions
}
import Macros.Annotations.DefaultValue

case class FooWithDefault2(
  id: Int,
  @DefaultValue("default") title: String)

{
  val reader: BSONDocumentReader[FooWithDefault2] =
    Macros.using[MacroOptions.ReadDefaultValues].reader[FooWithDefault2]

  reader.readTry(BSONDocument("id" -> 1)) // missing BSON title
  // => Success: FooWithDefault2(id = 1, title = "default")
}

The new @Reader allows to indicate a BSON reader to be used for a property, not using the implicit scope.

import reactivemongo.api.bson.{ BSONDocument, BSONReader }
import reactivemongo.api.bson.Macros, Macros.Annotations.Reader

val scoreReader: BSONReader[Double] = BSONReader.collect[Double] {
  case reactivemongo.api.bson.BSONString(v) => v.toDouble
  case reactivemongo.api.bson.BSONDouble(b) => b
}

case class CustomFoo1(
  title: String,
  @Reader(scoreReader) score: Double)

Macros.reader[CustomFoo1].readTry(BSONDocument(
  "title" -> "Bar",
  "score" -> "1.23" // accepted by annotated scoreReader
))
// Success: CustomFoo1(title = "Bar", score = 1.23D)

Also the new @Writer specifies the BSON writer to be used for a property, instead of the implicit scope.

import reactivemongo.api.bson.{ BSONString, BSONWriter }
import reactivemongo.api.bson.Macros,
  Macros.Annotations.Writer

val scoreWriter: BSONWriter[Double] = BSONWriter[Double] { d =>
  BSONString(d.toString) // write double as string
}

case class CustomFoo2(
  title: String,
  @Writer(scoreWriter) score: Double)

val writer = Macros.writer[CustomFoo2]

writer.writeTry(CustomFoo2(title = "Bar", score = 1.23D))
// Success: BSONDocument("title" -> "Bar", "score" -> "1.23")

Extra libraries

Some extra libraries are provided along the new BSON one, to improve the integration.

GeoJSON:

A new GeoJSON library is provided, with the geometry types and the corresponding handlers to read from and write them to appropriate BSON representation.

It can be configured in the build.sbt as below.

libraryDependencies ++= Seq(
  "org.reactivemongo" %% "reactivemongo-bson-geo" % "1.1.0-RC12")

See Scaladoc

Monocle:

The library that provides Monocle based optics, for BSON values.

It can be configured in the build.sbt as below.

libraryDependencies ++= Seq(
  "org.reactivemongo" %% "reactivemongo-bson-monocle" % "1.1.0-RC12")

See Scaladoc

Specs2:

The Specs2 library provides utilities to write tests using specs2 with BSON values.

It can be configured in the build.sbt as below.

libraryDependencies ++= Seq(
  "org.reactivemongo" %% "reactivemongo-bson-specs2" % "1.1.0-RC12")

See Scaladoc

Query and write operations

The query builder supports more options (see find).

The collection API provides new builders for write operations. This supports bulk operations (e.g. insert many documents at once).

InsertBuilder

The new insert operation is providing an InsertBuilder.

It supports simple insert with .one.

import scala.concurrent.Future
import scala.concurrent.ExecutionContext.Implicits.global

import reactivemongo.api.commands.WriteResult

import reactivemongo.api.bson.BSONDocument
import reactivemongo.api.bson.collection.BSONCollection

val document1 = BSONDocument(
  "firstName" -> "Stephane",
  "lastName" -> "Godbillon",
  "age" -> 29)

// Simple: .insert.one(t)
def simpleInsert(coll: BSONCollection): Future[WriteResult] =
  coll.insert.one(document1)

Bulk insert is supported by .many function.

import scala.concurrent.Future
import scala.concurrent.ExecutionContext.Implicits.global

import reactivemongo.api.bson.BSONDocument
import reactivemongo.api.bson.collection.BSONCollection

// Bulk: .insert.many(Seq(t1, t2, ..., tN))
def bulkInsert(
  coll: BSONCollection): Future[coll.MultiBulkWriteResult] =
  coll.insert(ordered = false).many(Seq(
    document1, BSONDocument(
      "firstName" -> "Foo",
      "lastName" -> "Bar",
      "age" -> 1)))

UpdateBuilder

The new update operation returns an UpdateBuilder. It supports simple update.

import scala.concurrent.ExecutionContext.Implicits.global

import reactivemongo.api.bson.BSONDocument
import reactivemongo.api.bson.collection.BSONCollection

def simpleUpdate(personColl: BSONCollection) = {
  val selector = BSONDocument("name" -> "Jack")

  val modifier = BSONDocument(
    "$set" -> BSONDocument(
      "lastName" -> "London",
      "firstName" -> "Jack"),
      "$unset" -> BSONDocument("name" -> 1))

  personColl.update.one(
    q = selector, u = modifier,
    upsert = false, multi = false)
}

It also allows to perform bulk update using the .many function.

import scala.concurrent.Future
import scala.concurrent.ExecutionContext.Implicits.global

import reactivemongo.api.bson.document
import reactivemongo.api.bson.collection.BSONCollection

def bulkUpdate(personColl: BSONCollection) = {
  val updateBuilder1 = personColl.update(ordered = true)
  val updates = Future.sequence(Seq(
    updateBuilder1.element(
      q = document("firstName" -> "Jane", "lastName" -> "Doh"),
      u = document("age" -> 18),
      upsert = true, multi = false),
    updateBuilder1.element(
      q = document("firstName" -> "Bob"),
      u = document("age" -> 19),
      upsert = false, multi = true)))

  updates.flatMap { updateBuilder1.many(_) }
}

DeleteBuilder

The .delete function returns a DeleteBuilder. It supports simple deletion.

The .remove operation is now deprecated.

import scala.concurrent.ExecutionContext.Implicits.global

import reactivemongo.api.bson.BSONDocument
import reactivemongo.api.bson.collection.BSONCollection

def simpleDelete1(personColl: BSONCollection) =
  personColl.delete.one(BSONDocument("firstName" -> "Stephane"))

Bulk deletion is supported with the .many function.

import scala.concurrent.Future
import scala.concurrent.ExecutionContext.Implicits.global

import reactivemongo.api.bson.BSONDocument
import reactivemongo.api.bson.collection.BSONCollection

def bulkDelete1(personColl: BSONCollection) = {
  val deleteBuilder = personColl.delete(ordered = false)
  val deletes = Future.sequence(Seq(
    deleteBuilder.element(
      q = BSONDocument("firstName" -> "Stephane"),
      limit = Some(1), // former option firstMatch
      collation = None),
    deleteBuilder.element(
      q = BSONDocument("lastName" -> "Doh"),
      limit = None, // delete all the matching document
      collation = None)))

  deletes.flatMap { ops => deleteBuilder.many(ops) }
}

arrayFilters

The arrayFilters criteria is supported by the findAndModify operation.

import scala.concurrent.ExecutionContext.Implicits.global

import reactivemongo.api.WriteConcern
import reactivemongo.api.bson.BSONDocument
import reactivemongo.api.bson.collection.BSONCollection

def findAndUpdateArrayFilters(personColl: BSONCollection) =
  personColl.findAndModify(
    selector = BSONDocument.empty,
    modifier = personColl.updateModifier(
      update = BSONDocument(f"$$set" -> BSONDocument(
        f"grades.$$[element]" -> 100))),
    sort = None, fields = None,
    bypassDocumentValidation = false,
    writeConcern = WriteConcern.Journaled,
    maxTime = None, collation = None,
    arrayFilters = Seq(BSONDocument(
      "elem.grade" -> BSONDocument(f"$$gte" -> 85))))

arrayFilters is also supported on update.

import scala.concurrent.ExecutionContext.Implicits.global

import reactivemongo.api.bson.BSONDocument
import reactivemongo.api.bson.collection.BSONCollection

def updateArrayFilters(personColl: BSONCollection) =
  personColl.update.one(
    q = BSONDocument(
      "grades" -> BSONDocument(f"$$gte" -> 100)),
    u = BSONDocument(f"$$set" -> BSONDocument(
      f"grades.$$[element]" -> 100)),
    upsert = false,
    multi = true,
    collation = None,
    arrayFilters = Seq(BSONDocument(
      "element" -> BSONDocument(f"$$gte" -> 100))))

The .count operation now returns a Long value (rather than Int).

import scala.concurrent.Future
import scala.concurrent.ExecutionContext.Implicits.global

import reactivemongo.api.bson.collection.BSONCollection

def countPerson(personColl: BSONCollection): Future[Long] =
  personColl.count(None/* all */)

WriteResult

A new utility is provided to extract exception details from an erroneous result.

import reactivemongo.api.commands.WriteResult

def printExceptionIfFailed(res: WriteResult) = res match {
  case WriteResult.Exception(cause) =>
    cause.printStackTrace()

  case _ =>
    println("OK")
}

Reminder: There is no need to check failure inside a Future[WriteResult] as soon as the Future is successful.

More: Find documents, Write documents

Play

Play integration has been upgraded, to support versions up to 2.8 and be compatible with the new BSON library.

The JSONCollection and JSONSerializationPack (from package reactivemongo.play.json.collection) have been removed, and JSON compatibility can be applied using standard collection and JSON conversions.

The play.modules.reactivemongo.JSONFileToSave has also been removed.

JSON compatibility

The JSON/BSON compatibility has been refactored. The main import is:

import reactivemongo.play.json.compat._

Considering the following User class:

package object jsonexamples1 {
  import reactivemongo.api.bson._

  case class User(
    _id: BSONObjectID, // Rather use UUID or String
    username: String,
    role: String,
    created: BSONTimestamp, // Rather use Instant
    lastModified: BSONDateTime,
    sym: Option[BSONSymbol]) // Rather use String

  object User {
    implicit val bsonWriter: BSONDocumentWriter[User] =
      Macros.writer[User]

    implicit val bsonReader: BSONDocumentReader[User] =
      Macros.reader[User]
  }
}

Then specific imports are available to enable conversions, according the use cases.

import reactivemongo.play.json.compat._

// Conversions from BSON to JSON extended syntax
import bson2json._

// Override conversions with lax syntax
import lax._

// Conversions from JSON to BSON
import json2bson._
Convert BSON to JSON extended syntax

Scala:

import _root_.play.api.libs.json._

import _root_.reactivemongo.api.bson._

// Global compatibility import:
import reactivemongo.play.json.compat._

// Import BSON to JSON extended syntax (default)
import bson2json._ // Required import

import jsonexamples1.User

val userJs = Json.toJson(User(
  BSONObjectID.generate(), "lorem", "ipsum",
  created = BSONTimestamp(987654321L),
  lastModified = BSONDateTime(123456789L),
  sym = Some(BSONSymbol("foo"))))

JSON output: (userJs)

{
  "_id": {"$oid":"..."},
  "username": "lorem",
  "role": "ipsum",
  "created": {
    "$timestamp": {"t":0,"i":987654321}
  },
  "lastModified": {
    "$date": {"$numberLong":"123456789"}
  },
  "sym": {
    "$symbol":"foo"
  }
}
Convert BSON to JSON lax syntax

Scala:

import _root_.play.api.libs.json._
import _root_.reactivemongo.api.bson._

// Global compatibility import:
import reactivemongo.play.json.compat._

// Import BSON to JSON extended syntax (default)
import bson2json._ // Required import

// Import lax overrides
import lax._

import jsonexamples1.User

// Overrides BSONWriters for OID/Timestamp/DateTime
// so that the BSON representation matches the JSON lax one
implicit val bsonWriter: BSONDocumentWriter[User] =
  Macros.writer[User]

// Resolved from bsonWriter
val laxJsonWriter: OWrites[User] = implicitly[OWrites[User]]

import _root_.reactivemongo.api.bson._
import jsonexamples1.User

val laxUserJs = laxJsonWriter.writes(User(
  BSONObjectID.generate(), "lorem", "ipsum",
  created = BSONTimestamp(987654321L),
  lastModified = BSONDateTime(123456789L),
  sym = Some(BSONSymbol("foo"))))

JSON output: (userLaxJs)

{
  "_id": "...",
  "username": "lorem",
  "role": "ipsum",
  "created": 987654321,
  "lastModified": 123456789,
  "sym": "foo"
}
Convert JSON to BSON

Considering the Street class:

package object jsonexamples2 {
 case class Street(
   number: Option[Int],
   name: String)
}

The BSON representation can be derived from the JSON.

import _root_.play.api.libs.json._
import _root_.reactivemongo.api.bson._

// Global compatibility import:
import reactivemongo.play.json.compat._

// Import JSON to BSON conversions
import json2bson._ // Required import

import jsonexamples2.Street

implicit val jsonFormat: OFormat[Street] =
  Json.format[Street]

// Resolved from jsonFormat
val bsonStreetWriter =
  implicitly[BSONDocumentWriter[Street]]

bsonStreetWriter.writeTry(
  Street(Some(1), "rue de la lune"))
// Success: {'number':1, 'name':'rue de la lune'}
Value converters

Using the provided value conversions, a JSON object can be passed wherever a BSON document is expected.

import reactivemongo.api.bson.BSONDocument

def expectDoc(document: BSONDocument) =
  println(s"doc = ${BSONDocument pretty document}")

// ---

import _root_.play.api.libs.json._
import _root_.reactivemongo.api.bson._

// Global compatibility import:
import reactivemongo.play.json.compat._

// Import JSON to BSON conversions
import json2bson._ // Required import

expectDoc(document = Json.obj("age" -> Json.obj(f"$$gt" -> 1)))
  // doc = { 'age': { '$gt': 1 } }

See documentation

Aggregation

The Aggregation Framework supports more stages.

An aggregation pipeline is now a list of stage operator(s), possibly empty.

addFields

The $addFields stage can now be used.

import scala.concurrent.ExecutionContext

import reactivemongo.api.bson._
import reactivemongo.api.bson.collection.BSONCollection

def sumHomeworkQuizz(students: BSONCollection) =
  students.aggregateWith[BSONDocument]() { framework =>
    import framework.AddFields

    List(AddFields(document(
      "totalHomework" -> document(f"$$sum" -> f"$$homework"),
      "totalQuiz" -> document(f"$$sum" -> f"$$quiz"))), 
      AddFields(document(
        "totalScore" -> document(f"$$add" -> array(
        f"$$totalHomework", f"$$totalQuiz", f"$$extraCredit")))))
  }

bucketAuto

The $bucketAuto stage introduced by MongoDB 3.4 can be used as bellow.

import scala.concurrent.ExecutionContext

import reactivemongo.api.bson._
import reactivemongo.api.bson.collection.BSONCollection

def populationBuckets(zipcodes: BSONCollection)(
  implicit ec: ExecutionContext) =
  zipcodes.aggregateWith[BSONDocument]() { framework =>
    import framework.BucketAuto

    List(BucketAuto(
      BSONString(f"$$population"), 2, None)())
  }.collect[Set]()

count

The $count stage counts the aggregated documents.

import scala.concurrent.Future
import scala.concurrent.ExecutionContext.Implicits.global

import reactivemongo.api.bson._
import reactivemongo.api.bson.collection.BSONCollection

def countPopulatedStates1(col: BSONCollection): Future[Int] = {
  implicit val countReader = BSONDocumentReader.from[Int] { doc =>
    doc.getAsTry[Int]("popCount")
  }

  col.aggregateWith[Int]() { framework =>
    import framework.{ Count, Group, Match, SumField }

    Group(BSONString(f"$$state"))(
      "totalPop" -> SumField("population")) +: List(
        Match(BSONDocument(
          "totalPop" -> BSONDocument(f"$$gte" -> 10000000L))),
        Count("popCount"))
  }.head
}

facet

The $facet stage create multi-faceted aggregations, which characterize data across multiple dimensions (or facets).

import reactivemongo.api.bson.collection.BSONCollection

def useFacetAgg(coll: BSONCollection) = {
  import coll.AggregationFramework.{
    Count, Facet, Out, UnwindField
  }

  Facet(Seq(
    "foo" -> List(UnwindField("bar"), Count("c")),
    "lorem" -> List(Out("ipsum"))))
  /* {
    $facet: {
      'foo': [
        { '$unwind': '$bar' },
        { '$count': 'c' }
      ],
      'lorem': [ { '$out': 'ipsum' } ]
    }
  } */
}

filter

The $filter stage is now supported.

import scala.concurrent.ExecutionContext.Implicits.global

import reactivemongo.api.bson.{ BSONArray, BSONDocument, BSONString }
import reactivemongo.api.bson.collection.BSONCollection

def salesWithItemGreaterThanHundered(sales: BSONCollection) =
  sales.aggregateWith[BSONDocument]() { framework =>
    import framework._

    val sort = Sort(Ascending("_id"))

    Project(BSONDocument("items" -> Filter(
      input = BSONString(f"$$items"),
      as = "item",
      cond = BSONDocument(f"$$gte" -> BSONArray(
        f"$$$$item.price", 100))))) +: List(sort)

  }.collect[List]()

replaceRoot

The $replaceRoot stage is now supported.

import scala.concurrent.Future
import scala.concurrent.ExecutionContext.Implicits.global

import reactivemongo.api.bson.BSONDocument
import reactivemongo.api.bson.collection.BSONCollection

/* For a fruits collection: {
   "_id" : 1,
   "fruit" : [ "apples", "oranges" ],
   "in_stock" : { "oranges" : 20, "apples" : 60 },
   "on_order" : { "oranges" : 35, "apples" : 75 }
}, ... */

def replaceRootTest(
  fruits: BSONCollection): Future[Option[BSONDocument]] =
  fruits.aggregateWith[BSONDocument]() { framework =>
    import framework._

    List(ReplaceRootField("in_stock"))
  }.headOption // Result: { "oranges": 20, "apples": 60 }

In ReactiveMongo the Atlas Search feature can be applied through the aggregation framework.

import scala.concurrent.{ ExecutionContext, Future }

import reactivemongo.api.bson.BSONDocument
import reactivemongo.api.bson.collection.BSONCollection

def foo(col: BSONCollection)(
  implicit ec: ExecutionContext): Future[List[BSONDocument]] = {

  import col.AggregationFramework.AtlasSearch, AtlasSearch.Term

  col.aggregatorContext[BSONDocument](
    pipeline = List(AtlasSearch(Term(
      path = "description",
      query = "s*l*",
      modifier = Some(Term.Wildcard) // wildcard: true
    )))).prepared.cursor.collect[List]()
}

slice

The $slice operator is also supported as bellow.

import scala.concurrent.ExecutionContext

import reactivemongo.api.bson._
import reactivemongo.api.bson.collection.BSONCollection

def sliceFavorites(coll: BSONCollection)(
  implicit ec: ExecutionContext) =
  coll.aggregateWith[BSONDocument]() { framework =>
    import framework.{ Project, Slice }

    List(Project(BSONDocument(
      "name" -> 1,
      "favorites" -> Slice(
        array = BSONString(f"$$favorites"),
        n = BSONInteger(3)))))
  }.collect[Seq](4)

Miscellaneous

Other stages are also supported.

Change stream

Since MongoDB 3.6, it’s possible to watch the changes applied on a collection.

Now ReactiveMongo can obtain a stream of changes, and aggregate it.

import reactivemongo.api.Cursor
import reactivemongo.api.bson.BSONDocument
import reactivemongo.api.bson.collection.BSONCollection

def filteredWatch(
  coll: BSONCollection,
  filter: BSONDocument): Cursor[BSONDocument] = {
  import coll.AggregationFramework.{ Match, PipelineOperator }

  coll.watch[BSONDocument](
    pipeline = List[PipelineOperator](Match(filter))).
    cursor[Cursor.WithOps]
}

More: Aggregation Framework

GridFS

The GridFS API has been refactored, to be simpler and more safe.

The DefaultFileToSave has been moved to the factory fileToSave.

Separate classes and traits DefaultReadFile, ComputedMetadata, BasicMetadata and CustomMetadata have been merged with the single class ReadFile.

As the underlying files and chunks collections are no longer part of the public API, a new function update is provided to update the file metadata. Also note the DB.gridfs utility.

import scala.concurrent.ExecutionContext.Implicits.global

import reactivemongo.api.bson.{ BSONDocument, BSONObjectID }

import reactivemongo.api.DB

def updateFile(db: DB, fileId: BSONObjectID) =
  db.gridfs.update(fileId, BSONDocument(f"$$set" ->
    BSONDocument("meta" -> "data")))

Monitoring

A new module is available to collect ReactiveMongo metrics with Kamon.

"org.reactivemongo" %% "reactivemongo-kamon" % "1.1.0-RC12"

Then dashboards can be configured, for example if using Kamon APM.

Graph about established connections

More: Monitoring

Administration

The operations to manage a MongoDB instance can be executed using ReactiveMongo. This new release has new functions for DB administration.

Ping:

The DB has now a ping operation, to execute a ping command.

import scala.concurrent.Future
import scala.concurrent.ExecutionContext.Implicits.global

import reactivemongo.api.DB

def ping(admin: DB): Future[Boolean] = admin.ping()

Breaking changes

The Typesafe Migration Manager has been setup on the ReactiveMongo repository. It will validate all the future contributions, and help to make the API more stable.

For the current 1.1.0-RC12 release, it has detected the following breaking changes.

Connection

Operations and commands

Core/internal

Suggest changes