Switch Theme

Reverse-Engineering Cookies with Ktor

dev kotlin

4/6/2025

thumbnail

12 min. read

Introduction

Tabroom is a website used by the Speech & Debate community to check tournament data, read ballots, look at judges, and basically serves as the primary hub of managing competitions. It was created by Chris Palmer as a hobby project and is now property of the National Speech & Debate Association.

As I’ve continued development on a TabroomAPI, which scrapes the HTML content of the website to provide readable data, I eventually found myself having to introduce an authentication suite for accessing new content on the website. Now, Tabroom has this neat little website called api.tabroom.com that has a lot of the same data as the main website, but in a much more usable format.

The only problem was that entire API… isn’t actually accessible to the public, nor is it widely documented outside of its OpenAPI specification. Most of its methods are for specifically marked accounts. This means that I was presented with a dilemma: Either drop the authentication suite, or somehow reverse-engineer the Set-Cookie header when logging in through the HTML page to retrieve a session ID.

And I’m no quitter, so here’s how I did it.

Prerequisites

TabroomAPI is built on top of Ktor, a Kotlin-based HTTP client and server framework. This means that I needed to use Ktor’s HttpClient to make the requests to the Tabroom API. If you don’t have Ktor set up, you can follow the Ktor documentation to get started.

On top of this, because it scrapes HTML content, I also needed to use the Jsoup and Ksoup libraries to parse the HTML content and extract the necessary values. Because it’s built on top of Ktor, it’s available on multiple plaforms, meaning I needed to find different libraries to parse the HTML content on different platforms, and use different Ktor engines. For example, on Android, it uses the OkHttp engine, while on iOS, it uses the Darwin engine, and on the JVM, it uses the Java engine introduced in JDK11. You can look at the full build.gradle.kts file to see the dependencies and how they are set up.

As a result, it uses its own Document and Element classes as wrappers built on top of these libraries.

// Minimalistic document library only for the things I need
internal class Document(
    val url: String,
    val html: String
) {

    val body: Element
        get() = querySelector("body") ?: throw IllegalStateException("Document does not have a body element")

    val bodyElements: String
        get() = body.innerHTML
            .replace(Regex("<script\\b[^>]*>([\\s\\S]*?)</script>"), "")
            .replace(Regex("<style\\b[^>]*>([\\s\\S]*?)</style>"), "")

    val head: Element
        get() = querySelector("head") ?: throw IllegalStateException("Document does not have a head element")

}

// Performs a CSS Query Selection on the document
internal expect fun Document.querySelectorAll(selector: String): List<Element>
internal fun Document.querySelector(selector: String): Element? = querySelectorAll(selector).firstOrNull()
internal fun Document.getElementById(id: String): Element? = querySelector("#$id")
internal fun Document.getElementsByClassName(className: String): List<Element> = querySelectorAll(".$className")
internal fun Document.inputValue(name: String): String? {
    val input = querySelector("input[name=$name]") ?: return null
    return input["value"] ?: input["checked"]
}

// Minimilastic element library only for the things I needs
internal class Element(
    val tagName: String,
    val innerHTML: String,
    val textContent: String,
    val attributes: Map<String, String>,
    val children: List<Element>
) {
    operator fun get(attribute: String): String? = attributes[attribute]
}

Here’s an example implementaton on the JVM:

import org.jsoup.Jsoup

private fun org.jsoup.nodes.Element.convert(): Element {
    return Element(
        tagName = tagName(),
        innerHTML = html(),
        textContent = text(),
        attributes = attributes().asList().associate { it.key to it.value },
        children = children().map { it.convert() }
    )
}

internal actual fun Document.querySelectorAll(selector: String): List<Element> {
    val doc = Jsoup.parse(html)
    return doc.select(selector).map { it.convert() }
}

Now that we have all of that settled, let’s get started.

Inspecting the HTTP Request

The first thing I needed to figure out what how it naturally did it. It’s pretty simple: Tabroom will send your username and password in plain text to the server, along with a SHA and Salt with the password. There were other parameters that were completely unrelated, but I didn’t need them. The important part was that the server would respond with a Set-Cookie header, which is what I needed to get the session ID.

Looking through the Firefox network tab, this was pretty clear:

insert request stuff

The TabroomToken value is a URL-encoded session ID, that is sent along with each header in a Cookie request to the website to signify that a user is logged in.

Retrieving the Token

This is what the HTML form looks like on Tabroom:

<form name="login" action="/user/login/login_save.mhtml" method="post" class="signin">
    <input type="hidden" name="salt" value="aBhZ7Ec3">

    <input type="hidden" name="sha" value="$6$aBhZ7Ec3$zymKdLO6lp4eBarEM2bvauJyuIAxHgUNKDF3mZFzlBA0HmA2WuTwAu2Y89akiPPSROLvU.RNC/uPRALVi6Ug4/">

    <div class="full centeralign flexrow martop">
        <span class="ninetenths marno padvertno padleft padright">
            <input type="text" id="username" name="username" class="noresize full" value="" autocomplete="on" placeholder="Email address">
        </span>
    </div>

    <div class="full centeralign flexrow">
        <span class="ninetenths marno padvertno padleft padright">
            <input id="password" name="password" class="noresize full" type="password" value="" placeholder="Password">
        </span>
    </div>

    <div class="full centeralign flexrow">
        <span class="ninetenths marno padvertno padleft padright">
            <input type="submit" value="Login" class="noresize full">
        </span>
    </div>
</form>

Before you say I “leaked anything” - the salt and sha values don’t actually have any real security value. They are unique to every browser tab and change when you refresh the page every time. Because Tabroom is open source, you can see that these values are completely random and are used to generate the Session ID. Of course to actually create the session ID, you need the login credentials, which I obviously can’t share. But, if you have a Tabroom account, you can easily get this information by inspecting the network tab in your browser.

In response, I created two Kotlin methods: One to get these randomly generated values, and the other to use my Tabroom Credentials to retrieve a Session ID.

// Use an engine from different platforms
internal expect val engine: HttpClientEngine

// Initialize our client
internal val client
    get() = HttpClient(engine) {
        expectSuccess = false
        followRedirects = false
    }

// Internal method to retrieve a document by its URL
internal suspend fun String.fetchDocument(useToken: Boolean = true): Document {
    val res = client.get(this) {
        headers {
            append("User-Agent", "Ktor HTTP Client, Tabroom API v1")

            append("Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8")
            append("Accept-Language", "en-US,en;q=0.9")
            append("Connection", "keep-alive")
            append("Upgrade-Insecure-Requests", "1")
        }

        if (isLoggedIn && useToken)
            cookie("TabroomToken", token!!) // Use the "Cookie" function to set the cookie
    }

    if (!res.status.isSuccess()) throw IOException("Failed to fetch document: ${res.status}\n${res.bodyAsText(Charsets.UTF_8)}") // Print out the error message

    val text = res.bodyAsText(Charsets.UTF_8)
    return Document(this, text)
}

// Platform-specific url encoding/decoding
expect fun encodeURL(url: String): String
expect fun decodeURL(url: String): String


/**
 * Whether the API has currently stored a Session ID.
 */
val isLoggedIn: Boolean
    get() = token != null

/**
 * The current Session ID for the API.
 */
var token: String? = null
    private set

private suspend fun getSaltAndSha(): Pair<String, String> {
    val document = "https://www.tabroom.com/user/login/login.mhtml".fetchDocument() // Read and Parse HTML
    val salt = document.inputValue("salt") ?: throw IllegalStateException("Salt not found in document")
    val sha = document.inputValue("sha") ?: throw IllegalStateException("SHA not found in document")

    return Pair(salt, sha)
}

/**
 * Logs in to Tabroom with the given username and password.
 *
 * This function performs a login request to Tabroom and stores the authentication token in the `token` variable.
 * Tokens represent the Session ID and are valid for 1,024 hours, or roughly 42 days.
 * 
 * @param username The username to log in with.
 * @param password The password to log in with.
 * @return True if the login was successful, false otherwise.
 * @throws IOException If the login request fails.
 */
@JsName("loginAsync")
@JvmName("loginAsync")
suspend fun login(username: String, password: String): Boolean {
    val (salt, sha) = getSaltAndSha()

    val res = client.submitForm(
        "https://www.tabroom.com/user/login/login_save.mhtml",
        parameters {
            append("tourn_id", "")
            append("key", "")
            append("salt", salt)
            append("sha", sha)
            append("category_id", "")
            append("return", "")
            append("username", username)
            append("password", password)
        }
    ) {
        headers {
            append("User-Agent", "Ktor HTTP Client, Tabroom API v1")
            append("Content-Type", "application/x-www-form-urlencoded")
            append("Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8")
        }
    }

    // Redirect value of 302 means sucess; Don't be fooled by a 200 or something else
    if (res.status.value != 302) throw IOException("Unexpected status code: ${res.status}")

    val cookie = res.setCookie() // Retrieve the value of 'Set-Cookie'
    if (cookie.isEmpty()) return false

    val tabToken = cookie.firstOrNull { it.name == "TabroomToken" }?.value // Get the value of 'TabroomToken'
    if (tabToken == null) return false
    if (tabToken.isEmpty()) throw IllegalStateException("TabroomToken is empty")

    token = decodeURL(tabToken) // Set the token variable to the decoded value
    return true
}

This is a pretty long chunk of code, but the comments should be self-explanatory. We first fetch the HTML document from the login page, and then parse the salt and sha values from the HTML. We then use these values to create a POST request to the login page with our username and password. If the request is successful, we retrieve the Set-Cookie header and extract the TabroomToken value, which is our session ID.

The login function returns a boolean value indicating whether the login was successful or not. If the login was successful, the token variable is set to the session ID, which can be used for subsequent requests to the Tabroom API.

Conclusion

Now that I have a proper login function, I can use it to authenticate with the Tabroom API and retrieve data from the website. This is a pretty simple example of how to reverse-engineer a cookie using Ktor, but it can be applied to other websites as well.

Some example applications are in the ginormous fetcher.kt file available on GitHub. I hope this article was useful to you on how we could reverse-engineer cookies using Ktor.