A journey into safer and cleaner code in Scala

with some real-life examples

Paweł Kołodziejczyk

Agenda

  1. Introduction
    • From scratch to final system
  2. Errors categories
  3. Config issues
    • default values
    • lazy vals

Agenda

  1. Any, Unit - type safe?
    • Protobuf and json serialization
    • Akka actors
    • toString vs show
  2. Implicit conversions

Agenda

  1. The same type or different?
    • Type aliases
    • Values classes
    • Tagged types
  2. Summary

From scratch to final system

  • Idea / business needs
  • High-level architecture
    • Domain-Driven Design if necessary
  • More detailed
    • communication between components
    • ...

From scratch to final system

  • Single component (microservice) architecture
  • and implementation - Scala
    • here we are
  • ...

Error categories

Error categories

Configuration

  • can't be validated at compilation time
  • everything that differs between deployments
  • stored in env variables according to 12factor principles
  • HOCON - most popular format in Scala world
    • .conf files

Configuration

Example .conf file - HOCON syntax

kafka {
  url = "0.0.0.0:9092"
  url = ${?KAFKA_URL}
}

other-section {
  year = 1222
  year = ${?THIS_IS_ENV_FOR_YEAR}

  a = ${ENV_B}

  b = "some string"
}
other-section.b = "another string"

Configuration

Default values + overriding in .conf files
  • immutable single assignment parameters are safer and simplier
  • easy to misspell variable name - with default values more difficult to catch

Configuration

Default values directly in Scala code 

  val config = ConfigFactory.load()

  val kafkaUrl = Try(config.getString("kafka.url"))
    .getOrElse("0.0.0.0:9988")

  val year = Try(config.getInt("other-section.year"))
    .getOrElse("2019")
can cause errors difficult to find

Configuration

let's dockerize application 

version: '3.3'

services:
  some-app:
    image: asdasdasd/app:1.2.3
    restart: always
    ports:
      - "9000:8080"
    environment:
      ABC: "845"
      KAFKA_URI: "0.0.0.0:9092"
      QUITE_LONG_ENV_NAME: 1222
      THIS_IS_ENV_OR_YEAR: 2016
      THIS_ONE_IS_EVEN_LONGER_ENV_NAME: 123456
Did you notice missplellings in the env names?

Lazy vals in config

typesafe config library example 

  val config = ConfigFactory.load()

  lazy val kafkaUrl = config.getString("kafka.url")
  lazy val year = config.getInt("other-section.year")
  lazy val isFoo = config.getBoolean("foo.enabled")
  lazy val x = ...

Lazy vals in config

  • can fail only on the 1st access
    • when exactly?
  • fail-fast strategy is better
    • moves possible failures to initialization stage
    • regular vals
    • maybe another library
      • e.g. pureconfig
      • doesn't throw exception
      • returns Either[Err, ConfigStructure]

Be careful with lazy vals

when using it in performance critical places

  • it may slow down application
  • underlying implementation isn't perfect
  • improvements in Scala3 and Dotty

Lazy without "lazy" keyword?

  • Objects in Scala are created lazily when they are referenced
    • similar to lazy vals
  • if Object is local scope
    • behaves exactly as lazy val

Lazy without "lazy" keyword?


object Settings {
  val config = ConfigFactory.load()

  val kafkaUrl = config.getString("kafka.url")
  val year = config.getInt("other-section.year")
  val isFoo = config.getBoolean("other-section.foo.enabled")
}
all values will be evaluated if any will be accessed 

  Settings.kafkaUrl // or just Settings

Lazy without "lazy" keyword?


class Settings(config: Config) {
  object kafka {
    val url = config.getString("kafka.url")
  }
  object otherSection {
    val year = config.getInt("other-section.year")
    val isFoo = config.getBoolean("other-section.foo.enabled")
  }
}
inner objects behaves exactly as lazy vals 

  settings.kafka.url
will not instantiate otherSection object

Any, Unit? Protobuf example

Let's assume some model 

sealed trait Command
case class CreateUser(id: Long,
                      name: String,
                      tags: Set[String]) extends Command
case class UpdateUser(id: Long,
                      name: String,
                      tags: Set[String]) extends Command
case class RemoveUser(id: Long) extends Command

Any, Unit? Protobuf example

  • google Protocol Buffers serialization
  • protobuf DSL -> generated code (Java/Scala)
  • generated code (Java/Scala) -> model (Scala)
    • translator needed

Any, Unit? Protobuf example


  val userCreatedMapping: PartialFunction[AnyRef, AnyRef] = {
    case CreateUser(id, name, tags) =>
      generated.CreateUser.newBuilder()
        .setUserId(id)
        .setName(name)
        .setTags(tags.asJava)
        .build()

    case cmd: generated.CreateUser =>
      CreateUser(cmd.id, cmd.name, cmd.tags.asScala)
  }

Any, Unit? Protobuf example


  val userCreatedMapping: PartialFunction[AnyRef, AnyRef] = {
    case s: String => "???"

    case CreateUser(id, name, tags) =>
      generated.CreateUser.newBuilder()
        .setUserId(id)
        .setName(name)
        .setTags(tags.asJava)
        .build()

    case cmd: generated.CreateUser =>
      CreateUser(cmd.id, cmd.name, cmd.tags.asScala)
  }

Any, Unit? Protobuf example


  val userCreatedMapping: PartialFunction[AnyRef, AnyRef] = {
    case CreateUser(id, name, tags) =>
      generated.CreateUser.newBuilder()
        .setUserId(id)
        .setName(name)
        .setTags(tags.asJava)

    case cmd: generated.CreateUser =>
      UpdateUser(cmd.id, cmd.name, cmd.tags.asScala)
  }
Did you notice what is wrong?

Any, Unit? Protobuf example


abstract class AbstractSerializer[ScalaType <: AnyRef,
                                  ProtosType <: AnyRef] {
  def convertFromScala(scalaType: ScalaType): ProtosType

  def convertFromProtos(protosType: ProtosType): ScalaType

  val mapping: PartialFunction[AnyRef, AnyRef] = {
    case scalaType: ScalaType =>
      convertFromScala(scalaType)

    case protosType: ProtosType =>
      convertFromProtos(protosType)
  }
}

Any, Unit? Protobuf example


object CreateUserSerializer extends
    AbstractSerializer[CreateUser, generated.CreateUser] {
  override def fromScala(cmd: CreateUser): generated.CreateUser =
    generated.CreateUser.newBuilder()
      .setUserId(cmd.id)
      .setName(cmd.id.name)
      .setTags(cmd.id.tags.asJava)
      // .build() // will be caught by compiler

  override def fromProtos(cmd: generated.CreateUser): CreateUser = {
    // incorrect type will be caught by compiler
    UpdateUser(cmd.id, cmd.name, cmd.tags.asScala)
  }
}

Any, Unit? Akka actors "Receive"


  type Receive = PartialFunction[Any, Unit]
  • receive method often grows in size
  • difficult to refactor legacy code

Any, Unit? Akka actors "Receive"


class UserActor extends Actor {
  override def receive: Receive = {
    case Create(...) => ...

    // ...
    // suppose a lot cases here

    case AddGroup(tag) => // do stuff

    case RemoveGroup(id) => // do stuff
  }
}
Is it safe to remove last 2 cases?

Any, Unit? Akka actors "Receive"

  • Akka Typed
  • Maybe you don't need actors
  • monix

Any, Unit? Akka actors "Receive"


  def handle: PartialFunction[Command, List[Event]] = {
    case AddTag(tag) =>
      // ...
      List(...)

    case RemoveTag(tag) =>
      // ...
      List(...)
  }

  override def receive: Receive = {
    case cmd: Command =>
      val events = handle(cmd)
      // ...
  }
Part of code extracted to increase type safety

One for Any type - json converter

  • json4s - Format needed
  • customs formats per concrete type available
  • DefaultFormats for Any type - reflection based

One for Any type - json converter


  implicit val formats: Formats = DefaultFormats
  // ...
  onComplete(futureResponse) {
    case Success(response) =>
      complete(OK -> response)
    case Failure(throwable) =>
      complete(InternalServerError -> throwable)
  }
response and status are implicitly serialized to json

One for Any type - json converter


java.lang.StackOverflowError: null
        at scala.collection.generic.Growable$class.loop$1(Growable.scala:52)
        at scala.collection.generic.Growable$class.$plus$plus$eq(Growable.scala:57)
        at scala.collection.mutable.MapBuilder.$plus$plus$eq(MapBuilder.scala:25)
        at scala.collection.generic.GenMapFactory.apply(GenMapFactory.scala:48)
        at org.json4s.TypeHints$class.serialize(Formats.scala:237)
        at org.json4s.NoTypeHints$.serialize(Formats.scala:297)
        at org.json4s.Extraction$.internalDecomposeWithBuilder(Extraction.scala:113)
        at org.json4s.Extraction$.addField$1(Extraction.scala:110)
        at org.json4s.Extraction$.decomposeObject$1(Extraction.scala:140)
        at org.json4s.Extraction$.internalDecomposeWithBuilder(Extraction.scala:228)
        at org.json4s.Extraction$.addField$1(Extraction.scala:110)
        at org.json4s.Extraction$.decomposeObject$1(Extraction.scala:140)
        at org.json4s.Extraction$.internalDecomposeWithBuilder(Extraction.scala:228)

One for Any type - json converter

workaround - String instead of Throwable 

  implicit val formats: Formats = DefaultFormats
  // ...
  onComplete(futureResponse) {
    case Success(response) =>
      complete(OK -> response)
    case Failure(throwable) =>
      complete(InternalServerError -> throwable.getMessage)
  }
maybe newer versions fixed this

One for Any type - toString

  • toString method - from Java Object
  • every class contains it no matter if it is useful

  List(1, 2).toString // List(1, 2, 3)
  Array(1, 2).toString // [I@34ce8af7
  (new {}).toString // $$anon$1@b684286

One for Any type - toString

  • cats Show typeclass as an alternative
    • conversions for types you actually want
    • conversions on other types - compilation error

One for Any type - toString


  case class User(id: Long, name: String)
  case class DoNotShowMe(name: String)

  implicit val userShow: Show[User] =
    Show.show("User: " + _.name)

  User(7, "Michał").show // "User: Michał"
  DoNotShowMe("private").show // doesn't compile as expected

Implicit conversions

  • implicit defs -> you don't see def call
  • implicit class -> more "explicit" code

  // implicit class
  val timeout: FiniteDuration =
    config.getDuration("timeout").asFiniteDuration

  // implicit def
  val timeout: FiniteDuration =
    config.getDuration("timeout")

Implicit conversions

  • with implicit def you don't see def call
      you can with IDE support
  • with implicit class you can see conversion at a first glance
  • do not use too many implicit conversions
    • code can easily become less readable

The same type or different?

getting range of entities from db 

  def find(limit: Int, offset: Int): Task[E] = {
    ...
    prepare(limit, offset, ...)
  }

  def find(limit: Int, offset: Int, ...): Task[E] = {
    ...
    prepare(limit, offset, ...)
  }

  ...

The same type or different?

  • common mistake - swap parameters of the same type
  • compiler can't catch such mistakes

  val limit = 5
  val offset = 100

  find(limit, offset, ...)
  find(offset, limit, ...)
  find(17, 8, ...)

The same type or different?

Type aliases 

  type Limit = Int
  type Offset = Int
  def find(limit: Limit, offset: Offset): Task[E] = ...

  val limit: Limit = 5
  val offset: Offset = 100
  type Age = Int
  val age: Age = 35

  find(limit, offset, ...) // still compiles
  find(offset, limit, ...) // still compiles
  find(17, age, ...) // still compiles

The same type or different?

Type aliases

  • Aliases are not type safe
  • Just another names for Int
  • people intuitively trust they are helpful

The same type or different?

Value classes


  case class Limit(value: Int) extends AnyVal
  case class Offset(value: Int) extends AnyVal
  def find(limit: Limit, offset: Offset): Task[E] = ...

  val limit: Limit = Limit(5)
  val offset: Offset = Limit(100)

  find(limit, offset, ...) // ok
  find(offset, limit, ...) // doesn't compile
  find(17, 12, ...) // doesn't compile

The same type or different?

Value classes

  • Guarantee type safety
  • At runtime they will be no overhead
    • they will be raw Ints
    • however sometimes runtime allocation will happen

  case class UserId(id: Long) extends AnyVal
  Array[UserId] // elements of array will be fully allocated

The same type or different?

Tagged types

  • Type safety
  • No runtime overhead
  • Implementations:
    • shapeless, scalaz, softwaremill ...

  type @@[+T, +U] = T with Tag[U]

The same type or different?

Tagged types


  trait OffsetTag
  type Offset = Int @@ OffsetTag
  @inline def Offset(i: Int): Offset = i.taggedWith[OffsetTag]

  trait LimitTag
  type Limit = Int @@ LimitTag
  @inline def Limit(i: Int): Limit = i.taggedWith[LimitTag]

  val limit: Limit = Limit(5)
  val offset: Offset = Limit(100)
  find(limit, offset, ...) // ok
  find(offset, limit, ...) // doesn't compile
  find(17, 12, ...) // doesn't compile

softwaremill/scala-common example

The same type or different?

Tagged types


  trait UserIdTag
  type Offset = Int @@ UserIdTag
  @inline def UserId(i: Int): UserId = i.taggedWith[UserIdTag]

  val id = UserId(123)
  Array(id)

can be used similar to value classes

Summary

  • it's profitable to move possible errors from runtime to compilation time
  • if it's impossible, then to application initialization stage
  • there is always the opposite direction, compilation time -> runtime
      example: 
      Option(nullable).get

Summary

  • it's better to know that something is wrong
    • than to believe that it's ok when actually it isn't
  • simple improvements can significantly make code safer and cleaner

Thank you