7
$\begingroup$

First of all thank you to everyone who has been participating in the reviews to help moderate this growing community!

On the Stack Exchange network, there is the concept of banning someone from reviewing. We don't yet have any guidelines on when we'd like to implement such a review ban, and since previous attempts to get the community's opinion on policy have not resulted in any community input, I'll suggest what I think should be done, and invite the whole community to weigh with their opinions. If no one writes any opposition, we will assume the community is okay with the proposal.

First I'll say that out of the 10 last questions to be reviewed without a hammer-close, all resulted in the "leave open" decision, except for one question (the second highlighted question below was decided by a hammer):

enter image description here

Therefore there's been perhaps someone or some people being a bit too fervent about initiating close-votes, which some reviewers might find to be a bit bothersome, and it may slightly hinder the experience of some of our new users who are contributing questions here only to become disappointed when they see that our community decides to be a bit unwelcoming to the question. But banning people from making the "initial" close-vote is another story, and the only thing I'm trying to address in this post is the question of banning close "reviews", meaning that someone else has already made the initial close-vote.

Now the above screenshot shows a mix of "close" and "leave open" votes. As for a policy for banning close-reviewers, let's start with an extremely lenient policy:

  • The only circumstance (for now) in which we would ban a user from close-reviews is if they only vote to close 100% of the time (and at least 10 times) for 6 months (meaning that despite the community as a whole feeling that many questions should be closed and even more questions should be left open, the user is doing nothing but close voting). The same goes for people voting 100% to "leave closed" on re-open votes.
  • Edit: after a couple of weeks, I'd like to add one more suspension reason. If a user knowingly violates a site-wide policy related to close-voting/reviewing, for example this policy, then it is a lot of work for us to un-do that action: specifically we have to campaign to get a diamond moderator or 5 community moderators to re-open the question, and that can be seen as an extremely irritating waste of time when there's so much other work for us to do right now. But let's still stay on the side of leniency here: if but only if a user knowingly violates a site policy, and does not make an effort to inform us of their reason for disagreeing with the policy (for example on the Meta post associated with the policy), should we suspend them from close-voting.

Also as further evidence of people being perhaps a bit trigger-happy with close voting, you can see that we have had 457 close-vote reviews as of 14 January 2022:

enter image description here

At the same time, Operations Research has only 368 (despite being about 1 year older than us and more than 400 questions more than us):

enter image description here

$\endgroup$
11
  • $\begingroup$ I think with how lenient the standard, I would be able to to get behind this. Would the same standard apply to the reverse situation? If a user only voted to "leave open" or "reopen", would that be grounds for a review suspension? I would think the argument for a suspension for always voting one way is that the user isn't really "reviewing"; they have already decided how they are going to vote on any question, regardless of the quality. $\endgroup$
    – Tyberius Mod
    Jan 26 '21 at 16:59
  • $\begingroup$ @Tyberius But the screenshot shows that only 1/10 of the last questions to finish review (without hammer) ended up being closed, so if someone voted to "leave open" 10 times in a row, they are only going against the grain a small percentage of the time. They might also not be showing up in the review because they are the first to vote close, or they are voting to close via flags in the case of abandoned questions. I haven't seen any problem with someone being too lenient with keeping things open, have you? My close votes come in the form of flags usually so they don't show up in the reviews. $\endgroup$
    – Nike Dattani
    Jan 26 '21 at 17:41
  • $\begingroup$ I just wanted to make sure I was understanding the scope of the proposal (i.e. whether we wanted to discourage reviews that are consistently out of step with a community philosophy or whether we wanted to discourage specifically poor reviews, where the reviewer always chooses a particular option). $\endgroup$
    – Tyberius Mod
    Jan 26 '21 at 20:11
  • 5
    $\begingroup$ I'm not sure what I think about this. The site is relatively new, so it's still not clear where to draw the line between what's on-topic and what's off-topic. Hence, someone could conceivably vote to close a lot (and skip on all other votes) without any ill intent. On the other hand, 100% is 100%... Second, I don't think one should read too much into the fact that only a small proportion of those questions get closed. I'd imagine that's an expected outcome on smaller stacks with few reviewers. $\endgroup$
    – Anyon
    Jan 26 '21 at 20:51
  • 1
    $\begingroup$ I am in the boat of thinking that some users will be more into closing than others. I don't think that the percentage of close to leave open votes is really too relevant, since people may have firm opinions on what doesn't go here vs what might go here. Same the other way around. $\endgroup$
    – Tristan Maxson
    Jan 27 '21 at 19:20
  • $\begingroup$ @Tyberius If we were to ban a user from close-reviewing for voting "leave open" 100% of the time for at least 10 votes, then based on the last 10 questions that went up for review, we would essentially be banning them for going against the community in only this case. They will have agreed with the community's overall decision in 90% of cases, so I'd be happy to revisit this later when there's far more reviewers and more review activity, but for now I think banning 100% close voters & not banning 100% leave-open voters makes sense. $\endgroup$
    – Nike Dattani
    Jan 27 '21 at 19:49
  • $\begingroup$ @Anyon Those are good points all around. The part about a small proportion of these questions actually getting closed is I think a signature of the fact that most of the questions going up for review, are probably not bad enough for our community to want them to be closed, because we have enough active close voters for those questions to be closed if they really were so bad. Instead of reading into the proportion of questions that get closed by votes, we can look at the screenshot which shows 11 close votes and 29 leave-open votes for 10 non-hammer questions. If someone close-voted all 10 $\endgroup$
    – Nike Dattani
    Jan 27 '21 at 19:59
  • $\begingroup$ questions, then I worry that they aren't really "reviewing", as Tyberius's first comment says. The close-vote ban would discourage such review behavior or act as a deterrent. @TristanMaxson That's a fair point indeed. I wonder what you think of this: The number of points needed to close vote is only 500, which isn't too hard to get. So what if someone earns 500 just by asking one question which becomes hot like this mattermodeling.stackexchange.com/q/3849/5, then close votes only, for the next 10 years? Should they still be able to close vote 10 years after they earn the 500 points? $\endgroup$
    – Nike Dattani
    Jan 27 '21 at 20:04
  • 2
    $\begingroup$ @NikeDattani I think that person should be able to close vote 10 years after they earn 500 points if its a reasonable close vote. I think someone repeatedly starting close votes which are not backed by the community requires some review of their votes, but as long as the community tends to align with them I don't see any reason that someone should be required to ever vote to keep open. They may skip and let it be someone else's problem. $\endgroup$
    – Tristan Maxson
    Jan 27 '21 at 20:07
  • $\begingroup$ "I think someone repeatedly starting close votes which are not backed by the community requires some review of their votes" -- I agree 100% with that, but the problem is that as far as I know, there is not any way to know who that person is. Even for moderators. Is that true @Tyberius? Now you say: "if its a reasonable close vote" and "as the community tends to align with them", but if someone is closing 100% of the time when only 1-10% of those questions do get closed, in my opinion the votes are not all "reasonable" and don't "align" with the community. $\endgroup$
    – Nike Dattani
    Jan 27 '21 at 20:15
  • $\begingroup$ We could have another discussion about what "reasonable" and "align with the community" means. $\endgroup$
    – Nike Dattani
    Jan 27 '21 at 20:18
0
$\begingroup$

Therefore there's been perhaps someone or some people being a bit too fervent about initiating close-votes

I don't see any argument or reasoning in the question from which this statement follows. How does '9 out of 10 posts in Review resulting in a "Leave Open" state' indicate that some people are too "Close"-happy? If anything, the data indicate the exact opposite (9 times out of 10 users have complete agreement on whether a question should be closed). Stifling different perspectives (in the form of actively suppressing the ability to review of users who don't agree with the majority 100% of the time), especially when they already aren't effecting a different outcome, is dangerous, and leads to groupthink.

I'll say that out of the 10 last questions to be reviewed without a hammer-close

How is closing via gold tag powers (e.g. hammer-close) different such that it's not a problem by this metric? Would your concern about the person/people voting to "Close" go away if they suddenly get 1k tag score? If the concern is justified (e.g. you're worried about someone casting the wrong vote or voting for the wrong reasons), then in my view the concern would only grow, because now said user(s) can close posts all by themselves.

If no one writes any opposition, we will assume the community is okay with the proposal.

This is not how Meta consensus is achieved on any other site on the network that I know of. A lack of engagement just means the post was not interesting enough to merit a response; it doesn't mean that everyone's onboard. If they were, they'd have upvoted the post. At the very least, you should be looking at the score of the Meta post (and any answers). Typically on Meta sites, something that only receives a handful of upvotes, net or gross, or has a near equal number of up vs down votes, is not indicative of a consensus. You need a score in the double digits at least, and with far more upvotes than downvotes.

To put it another way, there are 83 views on the question so far, and it has a score of 3. Obviously, some of these will be repeat viewers, some are your own views (as OP you can't vote on the post of course), and some might be visitors. But a good number of them (20k users and moderators may well be able to see more information about this) will be registered users on Matter Modeling who came to this post and, ostensibly, read it and decided it wasn't something they felt they should up vote.

Also, who is "we"?

The above concerns notwithstanding, review queue suspensions are "All or Nothing"... you can't suspend a user from only the Close Vote review queue, for example. You can only suspend someone from all review queues.

Considering MM is a newer, smaller site, there are likely to be exceedingly few questions in the Close Vote review queue, but a fairly impactful one in the First Posts, Late Answers, Suggested Edit, and perhaps even Low Quality Posts queue.

Moderators should take care that suspending someone for having a difference in opinion on a question's appropriate status re: open vs closed does not negatively impact their ability to help review in the other, more important and more populated queues.

$\endgroup$
8
  • $\begingroup$ There's been people close-voting questions that the rest of the community does not agree should be closed. There's been users consistently voting to close things in 100% of their reviews, and never voting for anything to re-open, despite the community deciding to keep those questions open nearly 100% of the time, and some of these users have contributed zero content to the site since the first week of Private Beta, so them having enough to close-vote might be controversial. About hammer closing, it only happens for one type of question for which the community agreed to have it hammer closed. $\endgroup$
    – Nike Dattani
    Feb 2 '21 at 22:58
  • 2
    $\begingroup$ @NikeDattani by hammer closing I assumed you meant people with gold badges, but it sounds like you mean when someone suggests a duplicate and the OP confirms it to self-close the question. If so, then that's something that happens completely outside of the Close Vote review queue. $\endgroup$
    – TylerH
    Feb 2 '21 at 23:04
  • 1
    $\begingroup$ About how we decide consensus in this community, that is how it's done: if a proposal is made on Meta and no one on Meta explicitly opposes it after long enough time, it's assumed there's consensus until someone opposes. We're not the only site with that policy. We also don't have 20k users, only 10% of that, and only 3 people with the Curious badge on Meta and only 8 people with the Commentator badge on Meta. $\endgroup$
    – Nike Dattani
    Feb 2 '21 at 23:04
  • 3
    $\begingroup$ @NikeDattani If a user is consistently voting to close 100% of things and never voting to reopen them, and more often than note runs afoul of the rest of the voters, that's a sign to me that moderators need to speak with the user privately and reassess the user's voting practices with them. If it is truly that egregious, then perhaps a review suspension is warranted, at least for a time. $\endgroup$
    – TylerH
    Feb 2 '21 at 23:05
  • 1
    $\begingroup$ Yes, Matter Modeling is a small, new site, but that doesn't mean the process is different, only the thresholds. $\endgroup$
    – TylerH
    Feb 2 '21 at 23:06
  • $\begingroup$ Zero people have gold badges on tags here. We don't even have enough questions on any tag to have someone with a silver badge. By "hammer closing" I mean that a moderator closed the question. This was done because the community agreed that certain questions should be closed. $\endgroup$
    – Nike Dattani
    Feb 2 '21 at 23:08
  • 1
    $\begingroup$ Yes in your penultimate comment, that's basically what I'm getting at. A user that is consistently voting the opposite way compared to everyone else. That's usually how people get review-banned in other sites too. We could have the moderators ask them privately about their behavior rather than suspending them from reviews right away, but a temporary ban on close-voting could also send the same message. The user in question would probably appreciate it to get the private message warning and chance to explain themselves, but it might also lead to more of a headache for the mods to deal with. $\endgroup$
    – Nike Dattani
    Feb 2 '21 at 23:13
  • 1
    $\begingroup$ Let's see what the mods think about whether to privately message them or temporarily ban them from close-voting. This Meta question alone might have caused the close-voters to re-consider their behavior, let's see what happens! $\endgroup$
    – Nike Dattani
    Feb 2 '21 at 23:14

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .