12 min. read
Tabroom is a website used by the Speech & Debate community to check tournament data, read ballots, look at judges, and basically serves as the primary hub of managing competitions. It was created by Chris Palmer as a hobby project and is now property of the National Speech & Debate Association.
As I’ve continued development on a TabroomAPI, which scrapes the HTML content of the website to provide readable data, I eventually found myself having to introduce an authentication suite for accessing new content on the website. Now, Tabroom has this neat little website called api.tabroom.com that has a lot of the same data as the main website, but in a much more usable format.
The only problem was that entire API… isn’t actually accessible to the public, nor is it widely documented outside of its OpenAPI specification. Most of its methods are for specifically marked accounts. This means that I was presented with a dilemma: Either drop the authentication suite, or somehow reverse-engineer the Set-Cookie
header when logging in through the HTML page to retrieve a session ID.
And I’m no quitter, so here’s how I did it.
TabroomAPI is built on top of Ktor, a Kotlin-based HTTP client and server framework. This means that I needed to use Ktor’s HttpClient
to make the requests to the Tabroom API. If you don’t have Ktor set up, you can follow the Ktor documentation to get started.
On top of this, because it scrapes HTML content, I also needed to use the Jsoup and Ksoup libraries to parse the HTML content and extract the necessary values. Because it’s built on top of Ktor, it’s available on multiple plaforms, meaning I needed to find different libraries to parse the HTML content on different platforms, and use different Ktor engines. For example, on Android, it uses the OkHttp
engine, while on iOS, it uses the Darwin
engine, and on the JVM, it uses the Java
engine introduced in JDK11. You can look at the full build.gradle.kts file to see the dependencies and how they are set up.
As a result, it uses its own Document
and Element
classes as wrappers built on top of these libraries.
// Minimalistic document library only for the things I need
internal class Document(
val url: String,
val html: String
) {
val body: Element
get() = querySelector("body") ?: throw IllegalStateException("Document does not have a body element")
val bodyElements: String
get() = body.innerHTML
.replace(Regex("<script\\b[^>]*>([\\s\\S]*?)</script>"), "")
.replace(Regex("<style\\b[^>]*>([\\s\\S]*?)</style>"), "")
val head: Element
get() = querySelector("head") ?: throw IllegalStateException("Document does not have a head element")
}
// Performs a CSS Query Selection on the document
internal expect fun Document.querySelectorAll(selector: String): List<Element>
internal fun Document.querySelector(selector: String): Element? = querySelectorAll(selector).firstOrNull()
internal fun Document.getElementById(id: String): Element? = querySelector("#$id")
internal fun Document.getElementsByClassName(className: String): List<Element> = querySelectorAll(".$className")
internal fun Document.inputValue(name: String): String? {
val input = querySelector("input[name=$name]") ?: return null
return input["value"] ?: input["checked"]
}
// Minimilastic element library only for the things I needs
internal class Element(
val tagName: String,
val innerHTML: String,
val textContent: String,
val attributes: Map<String, String>,
val children: List<Element>
) {
operator fun get(attribute: String): String? = attributes[attribute]
}
Here’s an example implementaton on the JVM:
import org.jsoup.Jsoup
private fun org.jsoup.nodes.Element.convert(): Element {
return Element(
tagName = tagName(),
innerHTML = html(),
textContent = text(),
attributes = attributes().asList().associate { it.key to it.value },
children = children().map { it.convert() }
)
}
internal actual fun Document.querySelectorAll(selector: String): List<Element> {
val doc = Jsoup.parse(html)
return doc.select(selector).map { it.convert() }
}
Now that we have all of that settled, let’s get started.
The first thing I needed to figure out what how it naturally did it. It’s pretty simple: Tabroom will send your username and password in plain text to the server, along with a SHA and Salt with the password. There were other parameters that were completely unrelated, but I didn’t need them. The important part was that the server would respond with a Set-Cookie
header, which is what I needed to get the session ID.
Looking through the Firefox network tab, this was pretty clear:
The TabroomToken
value is a URL-encoded session ID, that is sent along with each header in a Cookie
request to the website to signify that a user is logged in.
This is what the HTML form looks like on Tabroom:
<form name="login" action="/user/login/login_save.mhtml" method="post" class="signin">
<input type="hidden" name="salt" value="aBhZ7Ec3">
<input type="hidden" name="sha" value="$6$aBhZ7Ec3$zymKdLO6lp4eBarEM2bvauJyuIAxHgUNKDF3mZFzlBA0HmA2WuTwAu2Y89akiPPSROLvU.RNC/uPRALVi6Ug4/">
<div class="full centeralign flexrow martop">
<span class="ninetenths marno padvertno padleft padright">
<input type="text" id="username" name="username" class="noresize full" value="" autocomplete="on" placeholder="Email address">
</span>
</div>
<div class="full centeralign flexrow">
<span class="ninetenths marno padvertno padleft padright">
<input id="password" name="password" class="noresize full" type="password" value="" placeholder="Password">
</span>
</div>
<div class="full centeralign flexrow">
<span class="ninetenths marno padvertno padleft padright">
<input type="submit" value="Login" class="noresize full">
</span>
</div>
</form>
Before you say I “leaked anything” - the salt
and sha
values don’t actually have any real security value. They are unique to every browser tab and change when you refresh the page every time. Because Tabroom is open source, you can see that these values are completely random and are used to generate the Session ID. Of course to actually create the session ID, you need the login credentials, which I obviously can’t share. But, if you have a Tabroom account, you can easily get this information by inspecting the network tab in your browser.
In response, I created two Kotlin methods: One to get these randomly generated values, and the other to use my Tabroom Credentials to retrieve a Session ID.
// Use an engine from different platforms
internal expect val engine: HttpClientEngine
// Initialize our client
internal val client
get() = HttpClient(engine) {
expectSuccess = false
followRedirects = false
}
// Internal method to retrieve a document by its URL
internal suspend fun String.fetchDocument(useToken: Boolean = true): Document {
val res = client.get(this) {
headers {
append("User-Agent", "Ktor HTTP Client, Tabroom API v1")
append("Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8")
append("Accept-Language", "en-US,en;q=0.9")
append("Connection", "keep-alive")
append("Upgrade-Insecure-Requests", "1")
}
if (isLoggedIn && useToken)
cookie("TabroomToken", token!!) // Use the "Cookie" function to set the cookie
}
if (!res.status.isSuccess()) throw IOException("Failed to fetch document: ${res.status}\n${res.bodyAsText(Charsets.UTF_8)}") // Print out the error message
val text = res.bodyAsText(Charsets.UTF_8)
return Document(this, text)
}
// Platform-specific url encoding/decoding
expect fun encodeURL(url: String): String
expect fun decodeURL(url: String): String
/**
* Whether the API has currently stored a Session ID.
*/
val isLoggedIn: Boolean
get() = token != null
/**
* The current Session ID for the API.
*/
var token: String? = null
private set
private suspend fun getSaltAndSha(): Pair<String, String> {
val document = "https://www.tabroom.com/user/login/login.mhtml".fetchDocument() // Read and Parse HTML
val salt = document.inputValue("salt") ?: throw IllegalStateException("Salt not found in document")
val sha = document.inputValue("sha") ?: throw IllegalStateException("SHA not found in document")
return Pair(salt, sha)
}
/**
* Logs in to Tabroom with the given username and password.
*
* This function performs a login request to Tabroom and stores the authentication token in the `token` variable.
* Tokens represent the Session ID and are valid for 1,024 hours, or roughly 42 days.
*
* @param username The username to log in with.
* @param password The password to log in with.
* @return True if the login was successful, false otherwise.
* @throws IOException If the login request fails.
*/
@JsName("loginAsync")
@JvmName("loginAsync")
suspend fun login(username: String, password: String): Boolean {
val (salt, sha) = getSaltAndSha()
val res = client.submitForm(
"https://www.tabroom.com/user/login/login_save.mhtml",
parameters {
append("tourn_id", "")
append("key", "")
append("salt", salt)
append("sha", sha)
append("category_id", "")
append("return", "")
append("username", username)
append("password", password)
}
) {
headers {
append("User-Agent", "Ktor HTTP Client, Tabroom API v1")
append("Content-Type", "application/x-www-form-urlencoded")
append("Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8")
}
}
// Redirect value of 302 means sucess; Don't be fooled by a 200 or something else
if (res.status.value != 302) throw IOException("Unexpected status code: ${res.status}")
val cookie = res.setCookie() // Retrieve the value of 'Set-Cookie'
if (cookie.isEmpty()) return false
val tabToken = cookie.firstOrNull { it.name == "TabroomToken" }?.value // Get the value of 'TabroomToken'
if (tabToken == null) return false
if (tabToken.isEmpty()) throw IllegalStateException("TabroomToken is empty")
token = decodeURL(tabToken) // Set the token variable to the decoded value
return true
}
This is a pretty long chunk of code, but the comments should be self-explanatory. We first fetch the HTML document from the login page, and then parse the salt
and sha
values from the HTML. We then use these values to create a POST request to the login page with our username and password. If the request is successful, we retrieve the Set-Cookie
header and extract the TabroomToken
value, which is our session ID.
The login
function returns a boolean value indicating whether the login was successful or not. If the login was successful, the token
variable is set to the session ID, which can be used for subsequent requests to the Tabroom API.
Now that I have a proper login
function, I can use it to authenticate with the Tabroom API and retrieve data from the website. This is a pretty simple example of how to reverse-engineer a cookie using Ktor, but it can be applied to other websites as well.
Some example applications are in the ginormous fetcher.kt
file available on GitHub. I hope this article was useful to you on how we could reverse-engineer cookies using Ktor.