Private Application Reviews

[Mara]

This is a suggestion I'd like to bring up, based on some recent thinking I've done, along with seeing the opinions of various other roleplayers across the interwebs.


How about instead of staff/members/anyone posting whatever they like on pending apps out in public for everyone to read, we do private reviews? You know, send any notes or revisions in a PM to the member, and then continue the process that one.

It would de-clutter the app threads. I mean, some of those go to three pages or more, back and forth between staff and member. And it's kind of ridiculous.

Also, I think it would alleviate embarrassment of a lot of members who don't like to see their laundry aired out for everyone to read. I honestly think this is one of the reasons that new members leave once they receive a review on their app. (There are other reasons, but this is one of them).


I do admit that until recently, I just thought this was the way things were done. It's all I had encountered when putting up a character.

But in more recent years, I have encountered sites that once you finish a character and it's ready for review, they merely just slap on an "accepted" or a "pending--will PM you" if any edits are needed. And the ones needing fixing are very much in the majority. (But maybe their reviewing process is more lax than here).

And from speaking to others, people who run their own sites, browsing other sites for potential joinnage, I have noticed that there are very very few sites out there that still do public reviews.


There are pros and cons... but I really think the cons of public reviews outweigh the pros. And private reviews have much more pros than cons.

I know that if I had a choice, I would much prefer receiving a PM about any changes to an app I'd need to make.


So it's just an idea I'm throwing out there.  I know change can be fearsome, but if it makes the site better, I think it's something worth discussing.



PS: If the only reason to strike this down is because the staff believe that reviewing apps is the only way they can find new staff members... that's a weak excuse. Because potential staff are perfectly capable of PMing reviews to staff members, where they can decide whether or not they are qualified for the job. And being a mod is more than just reviewing anyway. There are so many other facets to it, facets that can be seen by just observing how members interact and write on the site. So no...  this is not a valid excuse, in my opinion, for keeping reviews public.



PPS:  I added in a poll, so that if anyone wants to express their opinion that way, they can.

[Thaddeus]

I voted to keep them public as firstly the process is more transparent: there can be no crossed wires as what everyone has said is a matter of public record unlike PMs which would require the releasing of the messages. Secondly I believe there is great utility for new people who wish to join SWU going through approved applications and see what reviewers have picked up on before - I deliberately picked the apps with the most comments to look at first so as to get an idea of what the staff and community see as issues, also it allows you to see how they behave.

For those two reasons I see no need to change the current review process. I will say though that I agree there is unnecessary clutter in reviewing applications, and when multiple staff respond it can have the affect of making a person feel under attack.

What I would suggest then instead of making it a private process is that the staff implement a process by which only one Moderator and one NGT member respond to each review at a time; with the NGT member responding first, acting as a filter. Members are encouraged to comment still, but only where relevant and they are not simply repeating someone else in the thread (when a staff or NGT gives a review there is no need for a member to come along and say what they said, it should be abundantly obvious that a person from one of those groups is qualified) and that there is no off-topic waffling.

There are my two cents.

Thadd

[Rugs]

Nov 8, 2014 4:09:55 GMT -5 Thaddeus said:

I voted to keep them public as firstly the process is more transparent: there can be no crossed wires as what everyone has said is a matter of public record unlike PMs which would require the releasing of the messages. Secondly I believe there is great utility for new people who wish to join SWU going through approved applications and see what reviewers have picked up on before - I deliberately picked the apps with the most comments to look at first so as to get an idea of what the staff and community see as issues, also it allows you to see how they behave

Pretty much this, yes. SWU is a complicated enough place as is, and keeping things open for all to see keeps the process pretty smooth. Occasionally we'll message people privately for getting things cleaned up after an initial review has been given, and I and other staffers are always happy to go into greater depth of discussion in private if needed, and I'd almost always end my reviews with some variation of "If you have questions/need something explained, feel free to send a message."

But it keeps what we're doing open and transparent, and also provides and maintains clear examples of precedent for anyone and everyone to see. I very much doubt Thaddeus is the only person who looks through reviews to see how we tick before hopping on the train.

And quite personally, as there have been periods where people in the past have thought the staff is some mal-intentioned cabal, I'm very strongly against privatizing the review process.

Yes, a review could be made public if needed, but at that point, there's not really any point to privatizing it in the first place.

[Fromikeable]

It would de-clutter the app threads. I mean, some of those go to three pages or more, back and forth between staff and member. And it's kind of ridiculous.

To be fair, the usual app rarely goes beyond a page or two. Really the only reason that happens is because, well, reviewing is a chatty process. I actually recall a few people PMing me back in the day about their apps before I was even an NGT, and I can say with all due certainty that the process was more complicated and a bit less coherent than a normal thread post. PM's seem a far more cluttering, far less efficient method to me, and it made those experiences about twice as hard as they needed to be. The reviewee had to hop back and forth between their app and their PM's just to converse with the reviewer, not to mention they risked losing any record of the process if they accidentally cleared out their mailbox one day. Keeping it all in one thread seems, to me at least, pretty compact and organized. It lets the record stand (especially since we don't delete threads if they're inactivated/abandoned), eliminates any deliberation down the line ("When I first made the app, she could sing, not dance!" prompts a look at the posts rather than a debate and scurrying through tons of PM's that may or may not have been kept), and keeps everything in one place. If one doesn't want to read the review, well... then one need not scroll past the actual app.

Also, I think it would alleviate embarrassment of a lot of members who don't like to see their laundry aired out for everyone to read. I honestly think this is one of the reasons that new members leave once they receive a review on their app. (There are other reasons, but this is one of them).

I question that idea, personally. People tend to post with full knowledge of a couple of key things:

A: Their work is public for all to see.

B: It's going to be subject to an application process i.e. peer review, suggestions and edits, etc.

C: Reviews are just meant to get the bio to be coherent enough for an outsider looking in and to match everything up with the site canon. Our goal isn't to roast people.

PS: If the only reason to strike this down is because the staff believe that reviewing apps is the only way they can find new staff members... that's a weak excuse. Because potential staff are perfectly capable of PMing reviews to staff members, where they can decide whether or not they are qualified for the job. And being a mod is more than just reviewing anyway. There are so many other facets to it, facets that can be seen by just observing how members interact and write on the site. So no... this is not a valid excuse, in my opinion, for keeping reviews public.

Again, PMing entire strings of reviews from PM sounds way less efficient to me. Beyond that, sure mods do more than just review, but I'd say our responsibility to get people RPing as soon as possible is probably our primary priority.


So to bundle that up in a sentence... just because someone else does it doesn't mean it works, or rather, that it works as well, or rather still, that it works as well in our particular format.

[Mara]

I think perhaps I hadn't explained myself well enough, or wires got crossed in the process.

The idea for having reviews PMed to people would not mean that they would ever be made public, except maybe in the staff forum if the reviewer wants opinions from other staff members on a particular atrocious app. But otherwise, it would function the same as the public reviews do now. Just that it would be one-on-one. I see no need for multiple people/mods to review the same app. Having just one person doing the same one, interacting with the member, would keep consistency throughout the process. (As I have seen from time to time in the past few years). All that would require is someone in the staff area going "I've got x's app" so that there is no doubling up.

How this would be more complicated than the current process, I don't understand.


Going off of that, I do agree with Thad's point that we should limit who responds to pending apps. I'd like to leave it to just one person, but I see that would make the NGT obsolete. So perhaps a compromise with the NGT still posting in the app threads themselves, but only once, to give a few observations/notes and leaving it at that. And later through PM a mod will give the actual review, a more in-depth review.

I am against letting regular members provide reviews. I think it just leads to chaos and confusion, as to who has authority and who doesn't. Plus, it'll keep things more contained if only NGT are allowed to reply to pending apps.

People tend to post with full knowledge of a couple of key things:

A: Their work is public for all to see.

B: It's going to be subject to an application process i.e. peer review, suggestions and edits, etc.

C: Reviews are just meant to get the bio to be coherent enough for an outsider looking in and to match everything up with the site canon. Our goal isn't to roast people.

Not necessarily. There are quite a few members who are unaware of our how unique review process works on SWU. Not everyone reads through a bunch of threads before they join. I personally know of at least a few who left the site because they were expecting a more private review process, and disliked that their app thread turned into a discussion bordering on argument. Or even if they did notice that it was out in the open, weren't expecting it to turn into the circus it did.




So I'm merely bringing up the possibility of changing things around here, so that we can do this, have a civil discussion about its merits and demerits.

[Stephen]

I have voted for "I don't care". If we do go to private reviews however, I feel that we should use dedicated reviewers. This would give back some sense of fairness that the lack of transparency would rob us of.

[Neology]

I do ask specific mods or NGT (or NGT hopefuls) to cover specific reviews, with the intention that the same person work with the applicant all the way until they're ready for a moderator approval stamp.

The only exception to this is when a staff member goes inactive, for whatever reason, and the application could not otherwise be reviewed in a timely manner.

To be honest, as far as non-staff posts on applications go, in the future I'd like to see a lot less partial reviews. ._. Just commenting on someone's bmi or hair color is not particularly helpful.

[Thaddeus]

Nov 8, 2014 12:57:29 GMT -5 Mara said:

The idea for having reviews PMed to people would not mean that they would ever be made public, except maybe in the staff forum if the reviewer wants opinions from other staff members on a particular atrocious app. But otherwise, it would function the same as the public reviews do now. Just that it would be one-on-one.

Again allow me to raise my first concern, but I will add some elucidation this time. As I mentioned the problem with private reviews is that it leaves things open to a lot of hearsay; for instance say someone feels a review they received in private was unfair, cruel, or unduly personal they could then talk about this to others (in cbox or via other mediums) creating a feeling of resentment or possibly worse. The only way to put things to rest would be for the staff to release the review which then defeats the purpose of private reviews. If you would have the staff respond to such situations by saying, "Sorry its SWU policy to have private reviews so we cannot comment on whether or not what person X is saying is true" then you are just inviting people to think of the staff as either being in the wrong or willfully ignoring the problem.

Nov 8, 2014 12:57:29 GMT -5 Mara said:

I see no need for multiple people/mods to review the same app. Having just one person doing the same one, interacting with the member, would keep consistency throughout the process. (As I have seen from time to time in the past few years). All that would require is someone in the staff area going "I've got x's app" so that there is no doubling up.

This is a concern you haven't raised before but I will respond again as I see an issue with this. Placing all the responsibility for a review (or more likely many reviews) on one person puts that person under undue stress I feel, and further it would slow down the review process more so than if there were two. My suggestion of using an NGT and Moderator would allow reviews to move reasonably quickly, as the NGT would keep things moving with regular responses and reviews, and also ensure that no one person feels like they are the only one pulling the load.

Nov 8, 2014 12:57:29 GMT -5 Mara said:

How this would be more complicated than the current process, I don't understand.

There would be no oversight of reviews unless again as I said the messages are released this is dramatically more complicated, and not only for the reasons I mentioned earlier. But also it means as reviewers we can get no feedback on the utility of our reviews without implementing another feedback process (i.e. posting the messages in the staff boards).

Nov 8, 2014 12:57:29 GMT -5 Mara said:

I am against letting regular members provide reviews. I think it just leads to chaos and confusion, as to who has authority and who doesn't. Plus, it'll keep things more contained if only NGT are allowed to reply to pending apps.

I may agree with you on this point, but at the same time I see the usefulness in having members reply and it is only usually the older members (such as yourself) or NGT in training that respond to applications as ordinary members. But I can as I said see the utility in restricting such individuals.

Nov 8, 2014 12:57:29 GMT -5 Mara said:

So I'm merely bringing up the possibility of changing things around here, so that we can do this, have a civil discussion about its merits and demerits.

Indeed, and that is good and I think everyone here is responding in the spirit of that notion community improvement. Thanks for bring it up, I look forward to the continuing discussion.

Thadd

[Rugs]

I am against letting regular members provide reviews. I think it just leads to chaos and confusion, as to who has authority and who doesn't. Plus, it'll keep things more contained if only NGT are allowed to reply to pending apps.

Everyone on our staff now and at least 90 percent of the staff we've had during my years at SWU got started as a normal member offering suggestions. That's something have and will continue to encourage. Any potential confusion is rarely any more than what comes out of the regular process of someone learning the ropes of doing reviews, and the only way you really learn reviews is by doing them, asking questions, and building from your mistakes as you grow. So unless we just start throwing people into the water without any training first (we won't) that's not really going to change. So that facet of SWU isn't going anywhere, quite frankly.

[Faeruy]

Not much I can add to that - except that I have modded at a forum where reviews were done privately, over PM, and frankly, it was awful. There was a complete lack of consistency, as nobody really knew how the other mods did their reviews. Also a complete lack of transparency - as someone new coming in, they had no idea what we were looking for in terms of apps. As someone who reviewed a lot, I know that I'm not perfect. I occasionally miss certain issues, or phrase things wrong. Having multiple pairs of eyes means fewer mistakes overall on our parts.

Also letting regular members review has it's advantages. Some of our 'regular' members are stronger in SW lore than I will ever be. Before I got promoted, I liked reviewing, in large part because it introduced me to a number of new characters and RP players that I normally wouldn't have gotten to know otherwise.

Frankly, anybody who spends five seconds looking around at the profiles (which they should do before starting one) will see how public the review process is. The review process has always been public here. I don't see any reason why it needs to change.