How we managed to used our user's power to keep our data clean
We had so many great users, who put their time and courage to create awesome reviews, photos and edit suggestions to venue's info, but the rate of spams and false info was gradually growing by the time.
We were previously launched our gamification platform which was trying to increase user's motivation by triggering their sense of pleasure and social acceptance by rewarding in-app consumable coins, special badges for super active users and also we had a leaderboard plan to show top users in each category/area. So these incentives was pretty awesome but on the other hand it was a motivation for fraud too.
That was a chance for us to make use of our loyal and caring users as super users or admins. They could audit our contents, users and more. By using them we could trigger their social acceptance senses too.
It is a common practice among many similar platforms like Foursquare to use their users' force to handle their quality of contents. There were also some very successful platforms in Iran which was using this solution.
We tried many concepts, to find a solution for how to select our super users or which set of behaviors imply that a user can be placed in a superuser segment, how can we invite them to be superuser, and also which set of tools can we give them to make their supervision tasks more facilitated.
With a wide competitor's benchmark, a couple of surveys and also interviews with our most loyal users, we came up with a superuser platform which had 10 different superuser levels, rules and a special dashboard for them to use.
We studied our user's behaviors with our BI team data scientists, we defined some key actions for our business and gave them weights based on their importance and effect on our platform. We built a formula to gave users a single score. Those factors was elements like their recency, approved contents, rejected contents, received coins amount etc. We called this score Dunro Citizen Score (DCS).
The DCS was a ruler for the platform and other Superusers to decide about someone's contributions. If the user is has so many rejected comments in a short period of time it will affect their DCS, and her/his future contributions will have less priority.
Our previous dashboard was just a simple set of pages with tabular data and buttons and it was used by our in-house operators. We wanted to have a public facing dashboard to simplify data audition very easy.
The was 4 types of data to get supervised:
I designed a special focus mode for each type of operation.
Below is a shot from the New Place Added dashboard, you can see who is the creator, what is his/her DCS, some stats about the panel (since we had this policy to instantly publish every place added and review it after, the amount of users checkins, comments and photos would be a great sign of the data accuracy), and the data provided by user.
In the left section we see the map and similar places. the reviewer can add corrections to each field and then decide to accept this place or not.
The Edit Suggestion dashboard had more complications, since the may be numerous suggestions by different users, the auditor should see a comparison view. Here is the panel:
We had a team of operators in our office which was regularly controlling our user generated contents, we put our superusers results in their hands to verify. We had an enormous change in their rate of editing and audition. They had more qualified contents to audit, they were also checking the rejected contents too. we had very positive reports from them. The superusers dashboard were used by over 30 people across the country to verify more than 120K comments, 200K New Places added and thousands of photos.
