home studios equipment staff/friends booking/rates for sale forum contact

Website: crapnotcrap.com

Vote and debate.

Moderators: kerble, Electrical-Staff

crapnotcrap.crap?

Crap
0
No votes
Not Crap
45
100%
 
Total votes : 45

Website: crapnotcrap.com

Postby var on Sun Jun 23, 2013 11:50 am

I started a weekend project to write a web crawler that crawls the CNC forum. I thought it'd be interesting to parse through all the data and get answers to the pertinent questions like 'what's the PRF's top 10 favorite movies' or 'what topics have the most replies/votes/views,' etc.

Stage 1 is complete, once a week it crawls the crap/notcrap forum and saves some basic information to a database. I registered a domain and threw a simple website on top of it that allows searching by keyword. It turns out that it's actually a decent search engine for crap/notcrap topics. I figured I'd share my results.

http://crapnotcrap.com

Update: Last weekend I wanted to learn Apple’s new programming language, Swift, so I revisited this project. I was also curious what the tools were like for beta testing and distributing an app on Apple's platform. I'm not entirely sure how some of you will feel about this, but the PRF, in some form, is now available on the iOS app store.

https://itunes.apple.com/us/app/crap-not-crap/id1109780693?mt=8
Last edited by var on Thu May 05, 2016 9:59 pm, edited 10 times in total.
User avatar
var
rocky mountain oysters
rocky mountain oysters
 
Posts: 461
Joined: Sun Mar 28, 2010 2:45 pm
Location: Philadelphia, PA

Re: Website: crapnotcrap.com

Postby warmowski on Sun Jun 23, 2013 12:12 pm

Clap clap clap clap clap clap clap clap clap clap clap clap clap clap clap clap clap clap clap clap whistle clap clap clap hoot.
User avatar
warmowski
Perfect Picture of Wisdom and Boldness
Perfect Picture of Wisdom and Boldness
 
Posts: 3072
Joined: Mon Feb 23, 2004 5:34 pm
Location: Chicago

Re: Website: crapnotcrap.com

Postby Podge9 on Sun Jun 23, 2013 12:13 pm

This is excellent. It's also a better search engine than the forum's own
User avatar
Podge9
roman
roman
 
Posts: 845
Joined: Tue Jun 17, 2008 10:45 am
Location: UK

Re: Website: crapnotcrap.com

Postby dabrasha on Sun Jun 23, 2013 12:15 pm

Doesn't work. This can't be right:

http://crapnotcrap.com/search/eddie%20money
User avatar
dabrasha
Master of Lit., Arts, & Architecture
Master of Lit., Arts, & Architecture
 
Posts: 3625
Joined: Tue Jan 24, 2006 11:33 pm
Location: Oakland

Re: Website: crapnotcrap.com

Postby andyman on Sun Jun 23, 2013 12:16 pm

absolutely splendid
offal wrote:I just want to be able to play bitchin' single-stroke rolls before I die
User avatar
andyman
Humankind's Greatest Musical Genius
Humankind's Greatest Musical Genius
 
Posts: 4324
Joined: Sat Apr 15, 2006 9:31 am
Location: Cambridge

Re: Website: crapnotcrap.com

Postby Boombats on Sun Jun 23, 2013 12:16 pm

Love the hench reference. Not Crap
Escape Rope / Blxck Onion / Black Mesa
warmowski wrote:Fire back, absolutely, always and forever no matter how much it blows Andy's mind.
User avatar
Boombats
Analingus Eggnog
 
Posts: 20705
Joined: Tue Oct 24, 2006 2:31 am

Re: Website: crapnotcrap.com

Postby Boombats on Sun Jun 23, 2013 12:20 pm

I like being able to compare the related polls' results!

Picture 4.png


Interesting that as a Renaissance man, VG is about 1/4 more crap than not crap, whereas in his performance as a musician, he is deemed about 1/3 more crap than not crap.
Escape Rope / Blxck Onion / Black Mesa
warmowski wrote:Fire back, absolutely, always and forever no matter how much it blows Andy's mind.
User avatar
Boombats
Analingus Eggnog
 
Posts: 20705
Joined: Tue Oct 24, 2006 2:31 am

Re: Website: crapnotcrap.com

Postby Frank Decent on Sun Jun 23, 2013 2:40 pm

Great work! Kind of fascinating.
User avatar
Frank Decent
Man with Encyclopedic Knowledge
Man with Encyclopedic Knowledge
 
Posts: 5274
Joined: Thu Jun 05, 2008 7:04 pm
Location: Canaduh

Re: Website: crapnotcrap.com

Postby Anthony Flack on Sun Jun 23, 2013 2:49 pm

I was idly considering the possibility of webcrawling this data too.

Find out the C/NC most and least controversial polls, based on vote split and number of total votes.

I guess it could maybe work something like this: a 100% not crap with only 10 votes would get a 90-100% NC ranking, or split the difference for 95%... whereas a 99% NC with 100 voters would be 98-99% NC rank or 98.5%.
Anthony Flack
Present-day God
Present-day God
 
Posts: 9595
Joined: Fri Aug 03, 2012 12:27 am
Location: New Zealand

Re: Website: crapnotcrap.com

Postby BClark on Sun Jun 23, 2013 4:36 pm

utterly brilliant.
I composed and produced the album California is Cold by Shahmen. Our song Poison has a video.
User avatar
BClark
Perfect Picture of Wisdom and Boldness
Perfect Picture of Wisdom and Boldness
 
Posts: 3093
Joined: Thu Dec 29, 2005 9:29 pm
Location: nomadic

Re: Website: crapnotcrap.com

Postby var on Mon Jun 24, 2013 9:20 am

Anthony Flack wrote:I was idly considering the possibility of webcrawling this data too.

Find out the C/NC most and least controversial polls, based on vote split and number of total votes.

I guess it could maybe work something like this: a 100% not crap with only 10 votes would get a 90-100% NC ranking, or split the difference for 95%... whereas a 99% NC with 100 voters would be 98-99% NC rank or 98.5%.


That's a good idea, I was also trying to think of ways to determine weekly changes in vote count so as to show a list of trending topics, but I have an idea it would be the same as whatever is on the front page. I'll probably release the code on github in a coulpe days if anyone wants to contribute.
User avatar
var
rocky mountain oysters
rocky mountain oysters
 
Posts: 461
Joined: Sun Mar 28, 2010 2:45 pm
Location: Philadelphia, PA

Re: Website: crapnotcrap.com

Postby llllllllllllllllllllllll on Mon Jun 24, 2013 9:30 am

This great! Also I guess this means I need to check out Unwound. Also Pimplemoose are some lousy cheats and fuck them. They need an asterisk next to their entry.
llllllllllllllllllllllllllllllllll

dvockins wrote:Gold hardware means you are number one.
User avatar
llllllllllllllllllllllll
Guardian Diety of the Planet
Guardian Diety of the Planet
 
Posts: 5888
Joined: Thu Jun 29, 2006 4:32 pm
Location: Texas

Re: Website: crapnotcrap.com

Postby warmowski on Mon Jun 24, 2013 9:45 am

All I'd suggest is please don't hammer the forum database too hard, as it already likes to take naps under typical load. If you're using curl for crawls, I'm sure you know the load-spreading switches and whatnot.

Looking forward to your github push. With this and other tools we could be witness to the birth of a new sociolinguistic subspecialty: prfology.

-r
User avatar
warmowski
Perfect Picture of Wisdom and Boldness
Perfect Picture of Wisdom and Boldness
 
Posts: 3072
Joined: Mon Feb 23, 2004 5:34 pm
Location: Chicago

Re: Website: crapnotcrap.com

Postby crustandcrumb on Mon Jun 24, 2013 10:23 am

you're a really useful engine, var. many thanks.
Sam wrote:i'm going to bootleg my own set with a Portastudio and upload/convert to .flac, burn and wrap the CD in a giant gatefold usb stick
User avatar
crustandcrumb
Saint Who Rules w/ Extensive Magnanimity
Saint Who Rules w/ Extensive Magnanimity
 
Posts: 2047
Joined: Sat Aug 16, 2008 9:31 pm
Location: Mid-Michigan

Re: Website: crapnotcrap.com

Postby Andy on Mon Jun 24, 2013 10:26 am

warmowski wrote:Looking forward to your github push. With this and other tools we could be witness to the birth of a new sociolinguistic subspecialty: prfology.

-r


I too would love to see the code.
User avatar
Andy
Leader with Extraordinary Personality
Leader with Extraordinary Personality
 
Posts: 2845
Joined: Sun Feb 29, 2004 12:04 pm
Location: 815

Re: Website: crapnotcrap.com

Postby Tommy on Mon Jun 24, 2013 10:31 am

Oh man, is this awesome! Great job.
User avatar
Tommy
Heaven-Sent Hero
Heaven-Sent Hero
 
Posts: 6077
Joined: Fri Sep 26, 2003 1:30 pm
Location: Logan Square, Chicago, IL

Re: Website: crapnotcrap.com

Postby Ptommydski on Mon Jun 24, 2013 4:39 pm

This is actually fantastic. Great work. Thanks a bunch for doing this, I will definitely use it!
User avatar
Ptommydski
Power Incarnate with Endless Creativity
Power Incarnate with Endless Creativity
 
Posts: 7455
Joined: Fri Oct 23, 2009 7:52 am
Location: UK

Re: Website: crapnotcrap.com

Postby Mr. Chimp on Mon Jun 24, 2013 4:49 pm

It's like you put everything into a bottle inside itself.


I drum Light Coma & Bully Pulpit & Andrew Cohen & Light Coma
I bass Small Awesome
User avatar
Mr. Chimp
Leader with Extraordinary Personality
Leader with Extraordinary Personality
 
Posts: 2777
Joined: Thu Nov 13, 2003 5:19 pm
Location: ChIL

Re: Website: crapnotcrap.com

Postby Dave//Eksvplot on Mon Jun 24, 2013 4:52 pm

These are useful, yes:

http://crapnotcrap.com/search/filmmaker%3A
http://crapnotcrap.com/search/director%3A

I forgot we did many of them.

Thanks.
madmanmunt wrote:You think Michelangelo Antonioni popped out of the womb with Monica Vitti in one hand and the reels of L'Avventura in the other?
User avatar
Dave//Eksvplot
Lode Star of the Twenty-First Century
Lode Star of the Twenty-First Century
 
Posts: 2333
Joined: Fri Jul 08, 2011 4:30 am

Re: Website: crapnotcrap.com

Postby Andy on Mon Jun 24, 2013 4:55 pm

I'd rather that the poll options weren't re-ordered. In some cases, the order of the options is significant.
User avatar
Andy
Leader with Extraordinary Personality
Leader with Extraordinary Personality
 
Posts: 2845
Joined: Sun Feb 29, 2004 12:04 pm
Location: 815

Next

Return to Crap / Not Crap

Who is online

Users browsing this forum: Google [Bot], TCracket and 15 guests