Decklists Will be back on July 8th! But in a much worse way.

Tyrant

New member
So, in a recent Twitter post on the magic online account (https://x.com/MagicOnline/status/1809258773070098646 ), they have announced that deck list will be back on the 8th but with the restrictions from before.


No prelims

only Top 32 of Challenges

a selection of 5-0s


Based on the tone, unwillingness to explain why, and daybreaks past actions, I am very doubtful this was a choice that Daybreak made. For those who are not aware Daybreak has been extremely open with the data in the past and for even a short time allowed us access to the api of all League data. I alongside many others were ecstatic for this change but now we won't even have all the 5-0s or challenge lists.


I'm making this thread less to just complain but to hopefully make WOTC aware of the large number of players and analysts that enjoy magic through the data. Wither that be talking about cool decks that 5-0, staying informed even when we aren't playing on metagames, or people like myself making spreadsheets to inform the players who just don't have the time to comb through the results. While I would personally love all the league data even just returning it to the state it was pre-API state that we had for months would be reasonable.


I ask that if you want the data back to use this thread to make WOTC aware just how much negative sentiment this change makes to both MTGO Users and the general player base, who loves to see results.
.
 

number9dream

New member
I'm a pretty casual MTGO player (read: losing player), big part of my engagement is enjoying following the competitive events. I can understand not displaying the full league data as it is kind of a privacy breach if you can look up someones exact results over time on MTGO, but not having full challenge data feels so silly when it's still available, just tedious to gather.

Not publishing every 5-0 is also a pretty big feel bad as a casual player - I've got exactly one 5-0, and despite being with a fairly unpopular deck, it didn't get published because someone else played a similar enough list. That kind of sucked, to be honest.

Roll-back to pre-API feels best, even if slightly disappointing as I was really looking forward to seeing the full data.
 

Qonfused

Member
I'm a pretty casual MTGO player (read: losing player), big part of my engagement is enjoying following the competitive events. I can understand not displaying the full league data as it is kind of a privacy breach if you can look up someones exact results over time on MTGO, but not having full challenge data feels so silly when it's still available, just tedious to gather.
I'm not sure if non-5-0 league decklists were supposed to be visible (though indeed you could query them). Adding to your point, instead of reworking leagues to curate 5-0 lists why not make only 5-0 lists visible? The privacy concern here would be much more diminished since you're only publishing results that were already at risk of being published through the 'curated' system.

More technically, the bottleneck for retrieving match wins for a league run was from obtaining a loginplayeventcourseid value that represents a player's league result, which you can only get from querying league decklists. To rectify this, you only need to restrict the decklists queryable for leagues, which also restricts all other league results. Compared to how league pages worked before, this also lowers the initial page-load time as the only lists queried (or possibly even scanned by a database) are the same 5-0 lists to be displayed.

[...] While I would personally love all the league data even just returning it to the state it was pre-API state that we had for months would be reasonable.
Roll-back to pre-API feels best, even if slightly disappointing as I was really looking forward to seeing the full data.
Just to clarify, the API was rolled out with the re-work of decklists way back on December 13th. There hadn't been any significant refinements to the API since the first few months it was rolled out. A creator had only recently discovered the API a few weeks ago (reference) which created some confusion about this in the Twitter circles.

At the very least I want to see full challenge data return. This is data that can be ascertained as-is through watching replays, so why hide it and make things harder?
Besides replays, you can also recover most of this information from just the standings of the tournament. The event standings themselves give you a great deal of information on tiebreakers and pairings, which still gives you a sizeable snapshot of the metagame even with only top-32 or x-1 decklists known. For larger tournaments, you'll often have users who scout the top 32 or 64 players to get decklists or archetype names and post them on Reddit. This was pretty much the status quo when decklists were posted a day later over half a year ago, and even older than that, when MTGO Dailies and other events were first hidden almost a decade ago.

However, this introduces another issue with the data -- we only know a tiny chunk of the top chunk of decklists, which means we can't observe a substantial bottom half of a deck's performance. This introduces a sizeable amount of survivorship bias, which combined with less total data means that the released results are quite misleading. There are a lot of emergent effects from this kind of bias, some of which are also helpful for reducing this bias, but for now I'll only cover the most visible effects to a player below by using a recent example.

A Trip Down Memory Lane

To illustrate this, we can compare the full Modern OTJ metagame (pre-MH3) with and without full event results:
520d0367-3f2e-420d-9ca3-37c1ac4fac7a
3f6c87d8-50ea-47af-8c93-7fee7f06c2a8

The first thing to notice is that we now know only a vanishingly small amount of decklists in the Top 32 or x-1 or better results. From the full results (left) to truncated (right), we go from 6,048 players to 570 players (less than 10% of the original dataset). In terms of matches, we go from a total of 30,424 matches to only 520; that's roughly 65 matches per week from what was originally 3,800 (only 2% of the original dataset). Additionally, we go from 92 unique archetypes to only 36.

We can also notice that both the top decks' metagame shares and winrates change quite substantially, especially for less popular decks. While we have a relatively comparable idea of what the top 5 decks are in both examples, we have relatively little information about the less popular archetypes. What's worse is that this creates an artificial 'gap' between archetypes metagame shares that grows exponentially, hurting the appeal of less played strategies that fall under the tail-end of the distribution of decks.

In both cases we observe what appears like two fairly solved metagames when looking left to right. We also still observe this same behavior on shorter timetables of only a couple weeks, which still presents a similar pool rankings for the top few decks. What this comparison shows is which strategies are hurt the most, which are the lesser-played decks that fall under much of the rest of the metagame.

This is much more observable when you flip the rankings of the two graphics to instead reflect the Top 32/x-1 or better results:
a187e5b8-5a02-4cc6-a1b9-41f288d434b5
cec0db64-e16c-4d16-bfae-6fd830d673f2

The second set of graphics show this to be a self-reinforcing pattern as truncating decklists increasingly widens the gap in the metagame shares presented without any player intervention needed. In other words, this is an effect that happens just from hiding the lower distribution of results. But players still use these truncated results to strategize for the next few or several tournaments, which turns into a feedback loop where only the top few decks make it past both "player-selection" and the tournaments themselves.

To better contextualize this point, below is a simulation that shows how winrates change with a top-32 cut of decklists:
07ee4a56-a3f1-42d4-b0f2-59fb94d58249


As only a small subset of the top decks remain salient features of the metagame after truncating results to top-32, this prevents break-out decks from further developing the metagame, dropping their playrate. What's worse is that truncating the lower distribution of results also inflates the winrates of the most played strategies (as shown in the graphic above), which further widens the gap in winrates presented. The more these gaps increase, the less likely it is for players to pick up other less played strategies and more likely it is for this loop to continue.

We've also observed this effect several times over the last several years after a large paper tournament publishes results. For example, we saw this happen both before and after Pro Tour Lord of the Rings nearly a year ago, where break-out decks from the Pro Tour broke a feedback loop in Modern from the past few weeks of MTGO results. For long stretches of time, the most played decks were synonymous with the most performing. We also saw the same pattern about a week ago at Pro Tour Modern Horizons 3, where the Modern metagame flipped from beating Ruby Storm to beating Nadu for the first time (also with only a partial albeit outdated view of MTGO results).

We can see that hiding this data doesn't actually slow down solving a metagame but instead hides the fact that the metagame isn't solved -- that there exists a wider pool of competitive strategies that can compete. Furthermore, it not only hides the diversity of other viable strategies in the metagame but stagnates their development by keeping break-out strategies out of results. Inevitably this leads to what players perceive as "solved" metagames much sooner than they should actually exist, though more critically, metagames are "solved" with less archetypes involved and thus less moving pieces to offset the balance.

An Open Letter to Daybreak

The information used to produce the above graphics brings us to is the least common denominator of a tournament: standings -- information needed for the competitive integrity of a tournament -- and decklists, which raises the question of why this data hiding is needed in the first place. With data hiding in place, players can still find the top several performing decks and observe metagames being "solved" in roughly equal or even faster periods of time, but what they can't observe is the wider pool of strategies the game offers to them. What I argue here is that it is not data analysis that stagnates metagames but data hiding.

I encourage Daybreak and the MTGO team to relay the same sentiment our community expresses to WotC, and to reconsider this change using the feedback in this thread to improve the API and continue to improve the game experience for players. With a game as dynamic and ever-changing as Magic: The Gathering, this decision reduces the pool of interesting cards and strategies to explore and interesting games of Magic accessible to many players, neither of which is an outcome the game creators intend.
 

GreyMerchant

New member
The sentiment toward these proposed regressive changes have been ABUNDANT on Twitter. Most of us are in agreement that this would be a step back in the wrong direction. It does feel like this is against the Daybreak ethos and more in line with WotC.

As people noted we had a very incomplete picture of the meta game at a point and to really have better information helps us all to make safer purchases and to also uncover more of the complexity that is behind our favourite formats. Your original post did not really mention any real harm but that it was "undesirable". That is not sufficient in this case. I am expecting a lot more information on this decision that is now coming like a radical u-turn.
 

death_lighter

New member
The current sentiment regarding the intent to only publish select 5-0 results, no prelim results and only the top 32 results is overwhelmingly negative.

It was really fantastic having access to entire challenge result decklists as well as ALL the 5-0s and I would strongly encourage you to go back to publishing these at the very least.
 

NoPainNoGame_

New member
Hiding informations to manipulate the meta? Not cool.
Probably wizard's management decided to make this move just because, ultimately, this will let them make a bit more money.
It's just sad.
The game we love is just a profit-machine for them.
I understand this is how the world goes, but damn, they could show some respect for the community in cases like this.
For me at least, i would be much more inclined to spend money on MTG if they were more respectful and JUST A BIT less profit-oriented.
This problem shows up regularly in every place MTG gets played (virtual or tabletop).
Greed is hurting magic, please do something about it wizards
 

Qonfused

Member
WotC is furious that Daybreak ruined the Pro Tour. They took away foil redemption and said to pray they don't take away more
MTGO results didn't show Nadu to be as dominant as it was at the Pro Tour; the metagame there was mostly based on teams' own preparation and the results of several paper tournaments after MH3 released. The API was taken down the week before the Pro Tour and wasn't made with knowledge of what decks players where choosing to bring. What was dominant on MTGO was Ruby Storm, which would have a more direct impact on players' preparation.
 

Qonfused

Member
Forgot to mention it on Sunday, but there's also been more conversation over on Reddit:
https://www.reddit.com/r/ModernMagic/comments/1dxnpmz/
https://www.reddit.com/r/magicTCG/comments/1dydnho/
https://www.reddit.com/r/spikes/comments/1dyda0r/

For those who are interested, here's a look at the impressions and analytics data over the last few days across Twitter and Reddit:
Twitter analytics data:
From the follow-up analysis tweet that covers the last 6 months of data:
be3fb09b-f528-43b6-9730-7ed124385755
From the original 'call-to-action' tweet that links to this forums page:
f2f5da4e-ea14-4b92-b6c5-01c69e8860c4

Reddit analytics data:
From r/ModernMagic (two days ago):
92f4eeb6-541d-4440-8eb0-6bc92e2c6c62
From r/magicTCG (repost; a day ago):
0cafd4a6-2423-4625-ad27-2a70676968fc
From r/spikes (repost; a day ago):
56970a4a-e97c-4f04-8504-63a57b263df5

It goes without saying that these posts have had quite the impact on the conversation online.
 
Top