Editing Project:Village pump (proposals) (section)

===An update after two weeks of discussion===
====1. What should the process look like?====
Hey all,

Thank you for your feedback. We recognize that this is a large and important change to our interfaces that will affect the default experience for millions of people. We appreciate your patience in this discussion on how to proceed in the best way possible for our readers, contributors, and communities.

We will try to summarize the feedback we have gotten so far, and continue with identifying next steps. Based on your feedback, we would like to propose the following process:

# Agree on what changes need to be made to the interface before the final deployment conversation
# Continue with a conversation focused on building consensus around deployment
# Deploy and continue with other improvements and requests that were agreed to be non-crucial for deployment

'''Does this seem in-line with your expectations?''' Do you have any concerns?

====2. Why are these changes improvements?====

Many of you were curios about the changes, and especially expressed interest in getting more details on our data and process. Below, we are outlining a bit about our process, as well as the data we have collected that proves that each feature is an improvement. Ping: @[[User:BilledMammal|BilledMammal]], @[[User:IAmChaos|IAmChaos]], [[User:Barkeep49|Barkeep49]], [[User:KevinL|KevinL]], [[User:Andrevan|Andrevan]].

TLDR: Every one of our changes goes through a rigorous process of research, development, qualitative and quantitative testing with readers and editors, prototype testing with editors (across 30+ language communities), iteration, and post-deployment monitoring. When a change does not meet the success criteria or does not perform better than the existing version of a tool, we either stop developing the change or iterate until performance is improved.

We believe that the changes we have made will be crucial to making the site more welcoming and easier to use to new readers and editors.

When compared to the older version of the Vector skin, Vector 2022 is proven to:

* Save time while reading and editing (measured by decreases in scrolling to the top of the page after the introduction of sticky header and table of contents)
* Increase exploration within a given page (measured by increased clicks to the table of contents)
* Improve readability (measured via collapsible sidebar usage)
* Improve the discovery of new content (measured by an increase in searching)

You will find more details in the section [[#UX research and usability testing]]. We have tweaked the existing comment a bit.
:Thank you for spending the time to write this out. My concern is that you are focused too much on testing for the change you intended to make and in doing so miss the broader impact. Because of this I feel your analysis only proves that you brought about your intended change, rather than proving {{tq|that each feature is an improvement}}.
:For example, consider the sticky header. Here, the goal is to (1) improve the user experience, by (2) saving reader time, by (3) reducing the amount of scrolling to the top that they need to do. 
:However, you only test for (3); you then infer (2) from the positive result for (3), and infer (1) from the positive result for (2). Testing directly for user experience is difficult, but to reduce the risk of errors the goal should be to get as close to that level as possible, and in this case it means testing for (2) rather than (3); if you look at #3 in my previous comment you will see that I am requesting data that should give us an answer to (2).
:In addition, you assume there are no negative impacts from the change. This isn't a safe assumption, and with #2 and #3 of my previous comment I request data that will allow us to consider this possibility. [[User:BilledMammal|BilledMammal]] ([[User talk:BilledMammal|talk]]) 06:56, 28 July 2022 (UTC)

:Hey @[[User:BilledMammal|BilledMammal]] - sorry for the delay in response!  I replied briefly to your initial comment above.  TLDR is that we try to design experiments using more precise metrics because more open-ended metrics (such as time spent on page) could be interpreted in multiple ways.  More time on page could be an improvement (people have a better experience and thus spend more time on the site overall).  Less time on page could also be seen as an improvement (we're saving people time in scrolling so they get to what they need to do quicker). We can keep discussing there or under this thread - whatever is easier! [[User:OVasileva (WMF)|OVasileva (WMF)]] ([[User talk:OVasileva (WMF)|talk]]) 09:11, 2 August 2022 (UTC)
::Thank you; no worries about the delay in response, and apologies for my own delay in response. I've replied above. [[User:BilledMammal|BilledMammal]] ([[User talk:BilledMammal|talk]]) 16:46, 7 August 2022 (UTC)

====3. Why are we having this conversation while development is still happening?====

We are eager to bring the improvements mentioned above to our readers. Currently, many new readers do not feel welcome by the interface as it is, and this is something we hope to solve as soon as possible. We recognize that no feature or skin will ever be perfect, and there will always be room for improvement. As we mentioned above, the skin, in its current form, is already a significant improvement over the current state.

The final state of the skin also depends on the conversation we’re having right now. There are many possible improvements or ideas for changes we can build and focus on. We’d like to discuss ''which of these are most important'' to the community as we proceed to implement and put the last touches on this version of the skin.

Finally, as we mentioned in a previous post, once deployed, we plan on continuing to work on the desktop experience. This project opens more possibilities for the future and gives us the opportunity to work with communities to provide new and necessary tools both for readers and editors. This is an ongoing process and it will be done with the feedback and collaboration of the community here and across other projects.

====4. What changes will be made before deployment?====

Our request for you is to '''review the list below and let us know if it looks correct in your opinion'''. What should be added? What should be removed? Do you have any questions on what each of these items will and will not include?

As a part of these conversations, we plan on placing requests into three categories. This categorization is based on our research, previous conversations with communities and prototype testing, as well as the feedback we received from all of you last week. These categories are flexible. We need your feedback to move something from one category to another, as well as to add items to each category. 

# '''Issues we would like to address prior to the deployment'''
## '''Table of Contents collapsing and narrow screens''' behavior (@[[user:xaosflux|xaosflux]], @[[user:sdkb|sdkb]]). We are working on this and hope to have it ready within the next few weeks (more details in [[phab:T306660|T306660]])
## '''Visual refinements''' (@[[user:IAmChaos|IAmChaos]], @[[user:Terasail|Terasail]]). We are working on this and we will finish before deployment, with the first part landing next week (week of August 1). To see more details on what visual refinements we are and how we worked with communities to define these, please see [[mw:Reading/Web/Desktop_Improvements/Features/Visual_Refinements|this page]]
## Making a decision on '''ToC handling and magic words '''(ping @[[User:Xaosflux|xaosflux]], @[[user:izno|izno]], @[[user:IAmChaos|IAmChaos]], @[[user:Anarchyte|Anarchyte]]). We are doing a more in-depth review of magic words and hope to come to you all with a proposal on what (if anything) we think would be best to change. Our sense is that for some of these use cases, the new ToC has solved the initial issues for them existing. We’re interested in finding out which use cases this is not the case for, and providing a solution for those. To confirm, however, the <nowiki>__NOTOC__</nowiki> magic word will continue to work, as will the templates creating the ToC based on <nowiki>__NOTOC__</nowiki> such as {{tl|horizontal toc}}
## '''Coordinates display''' and other [[Help:Indicator|indicator]] issues. We would like to ensure that coordinates do not overlap with any existing indicators and that the area in the top right corner of the article is well-organized ([[phab:T281974|T281974]]). Special thanks to Xaosflux, Izno, theDJ, Sdkb, AlexisJazz for participating in the discussion and helping us identify next steps. The conversation around coordinates continues in [[Wikipedia:Village pump (technical)#Coordinates in Vector 2022|VPT#Coordinates in Vector 2022]]
## '''Making it easy to opt-in and opt-out''' (@[[user:Shearonink|Shearonink]], @[[User:Wbm1058|wbm1058]]) - we have a button in the sidebar, which allows for easy recognition of opting out. Opting in is, however, only available through the preferences page. We’d like to explore the possibility of running banners that explain that the change was made and provide opt-out instructions as well. Similarly, prior to the change, we’d like to run more banners that encourage people to opt in and give us feedback
# '''Issues we would like to address after the deployment'''
## ToC/sidebar length and '''the [[mw:Reading/Web/Desktop_Improvements/Features/Page_tools|separation of page tools from wiki-wide tools]]''' (@[[user:sdkb|sdkb]]). This is a significant change that we would like to move forward with once we have everyone using the new default. This will be the best way to study and build out customizations for the various use cases (example: the ability to add admin tools or gadgets like [[wp:Twinkle|Twinkle]] to the menu)
# Issues that will not be addressed at this time (issues that are not part of the Desktop Improvements project, belonging to other teams, etc.)
## A preference which allows the fixed width of the page to be turned off.
##: <small>A local gadget (currently experimental) or personal userscripts may address this at an individual editor level. — [[User:Xaosflux|<span style="color:#FF9933; font-weight:bold; font-family:monotype;">xaosflux</span>]] <sup>[[User talk:Xaosflux|<span style="color:#009933;">Talk</span>]]</sup> 10:10, 28 July 2022 (UTC) </small>

====5. What should the deployment conversation look like?====

Some of you said that an RfC would be the best approach for the conversation around deployment. Does that sound like the right course of action? One thing that we have been thinking about is ways to include the voices of readers into the decision making process. We are planning to run surveys asking readers what they think about the new skin compared to the old one. How can we incorporate their thoughts into the conversation?

[[user:OVasileva (WMF)|OVasileva (WMF)]], [[User:SGrabarczuk (WMF)|SGrabarczuk (WMF)]] ([[User talk:SGrabarczuk (WMF)|talk]]) 01:53, 28 July 2022 (UTC)

:Amazing work! [[User:CactiStaccingCrane|CactiStaccingCrane]] ([[User talk:CactiStaccingCrane|talk]]) 05:31, 28 July 2022 (UTC)
:You could run a banner that provides an opt-in button for the new skin only visible for unregistered users, but then I'm not sure how they'd be able to leave feedback anonymously. The RfC could then be run in parallel with this campaign, with the ultimate decision relying on the inputs from both. [[User:Anarchyte|<span style="color:#202122;font-family:Trebuchet MS">Anarchyte</span>]] ([[User talk:Anarchyte|<span style="color:#202122">talk</span>]]) 06:03, 28 July 2022 (UTC)
::{{tq|with the ultimate decision relying on the inputs from both}} How would this work? Who gets to decide how much to weight each if they conflict? I don't foresee the community willingly relinquishing control, so as much as I'm concerned that the community isn't going to follow [[WP:READER]] and is going to [[Wikipedia:Requests_for_comment/2020_left_sidebar_update#A_note_on_power_users,_usability,_and_systemic_bias|overprivilege editor needs]], I think BilledMammal's suggestion is the only practical way to take reader input into account. <span style="color:#AAA"><small>&#123;{u&#124;</small><span style="border-radius:9em;padding:0 5px;background:#088">[[User:Sdkb|<span style="color:#FFF">'''Sdkb'''</span>]]</span><small>}&#125;</small></span> <sup>[[User talk:Sdkb|'''talk''']]</sup> 04:27, 29 July 2022 (UTC)
:::I echo {{u|Sdkb}}'s concerns. The skin should only be implemented if there is an affirmative consensus to switch to the new skin, especially since the new skin (as it currently stands) ''will'' make breaking changes to Wikipedia editor tools and Wikipedia article layouts. — [[User:Mhawk10|Ⓜ️hawk10]] ([[User talk:Mhawk10|talk]]) 23:41, 29 July 2022 (UTC)
:To incorporate their thoughts I would suggest running the surveys prior to the RfC, and allowing editors to assess the results of the survey when making their decision. In the end, this needs to be based on the consensus of the community, as assessed by the community. For this assessment I would suggest one thing; [[WP:RFCL|asking for a panel close]], such as was done for [[Wikipedia:Requests for adminship/2021 review/Proposals|the 2021 review of RfA]]. [[User:BilledMammal|BilledMammal]] ([[User talk:BilledMammal|talk]]) 07:23, 28 July 2022 (UTC)
::I think it's premature to decide in advance that multiple closers are needed. Most discussions can be evaluated adequately by a single closer. The 2021 RfA review was by design open-ended in the number of proposals that could be made, and thus unconstrained in the variety of rationales, which were primarily opinion-based, since often there's no way to collect data without actually trying a change to the process. It remains to be seen if an RfC on a new skin will have these or other characteristics such that more than one closer might be desirable. Ideally, if the development process goes as planned, there won't be an RfC unless there is already widespread support for the changes in question, much like the default deployment of the reply tool feature. [[User:Isaacl|isaacl]] ([[User talk:Isaacl|talk]]) 07:45, 28 July 2022 (UTC)
:::My suggestion was mainly to reduce the chance of the close being appealed at AN; I don't want us to have a situation where the discussion is closed with a consensus to implement the change, only to have the AN overturn the close. Normally such a situation would not be problematic but in this case I believe it would be due to the scale, prominence, and technical nature of the change. [[User:BilledMammal|BilledMammal]] ([[User talk:BilledMammal|talk]]) 08:04, 28 July 2022 (UTC)
::I think I agree with BilledMammal that the best way is to present the results of the surveys ''at'' the RfC. [[User:OVasileva (WMF)|Olga]], you really don't want to end up in the scenario where the enwiki community reaches a consensus ''against'' the change while readers say they like the change{{snd}}I think the community will feel betrayed if you don't respect its decision. If you don't think you can commit to abiding by the outcome of an RfC, you shouldn't hold an RfC; I would be quite upset but less than if you ran an RfC and then overruled its outcome. Best, '''[[User:L235|KevinL]]''' (<small>aka</small> [[User:L235|L235]] '''·''' [[User talk:L235#top|t]] '''·''' [[Special:Contribs/L235|c]]) 17:55, 2 August 2022 (UTC)