13 replies to this topic

[sjxx]

Hi,
I am looking for a script that will connect to a https webpage and enter in a set of data read from a .txt file, hit Submit, and do it all over again except now from the next line of data in the .txt file.

I tried building my own but it was very unsuccesful. I did some research regarding the WWW::Mechanize which should be the Perl object I need to use but this can not be used in a javascript form. The form that the script needs to enter the data in is in javascript so :/

any suggestions/ideas on how I can create something like this? I've searched google but nothing comes even close to my request.

Thanks

[myth]

http://search.cpan.o....800/lib/LWP.pm

# Create a user agent object  use LWP::UserAgent;  $ua = LWP::UserAgent->new;  $ua->agent("MyApp/0.1 ");  # Create a request  my $req = HTTP::Request->new(POST => 'http://search.cpan.org/search');  $req->content_type('application/x-www-form-urlencoded');  $req->content('query=libwww-perl&mode=dist');  # Pass request to the user agent and get a response back  my $res = $ua->request($req);  # Check the outcome of the response  if ($res->is_success) {      print $res->content;  }  else {      print $res->status_line, "\n";  }

http://www.cs.mcgill...erl/howto/open/

open( FILE, "< $filename" ) or die "Can't open $filename : $!";    while( <FILE> ) {        print;    }

So you'd access the line file using the variable $_. Depending on how your text file is setup, you'd need to split on ' ' or csv or similar using something like:

my ($username, $password) = split (/ /, $_)

So ultimately you'd have something like this (havent checked for syntax and this is really ugly / hacky)

#!/usr/bin/perl -wuse LWP::UserAgent;my $username;my $password;open( FILE, "< /tmp/blah.txt" ) or die "Can't open $filename : $!";    while( <FILE> ) {        print "Attempting line: " . $_;        ($username, $password) = split (/ /, $_);        k_go_now($username, $password);               # Sleep to keep us slightly more hidden and load off their servers        sleep(1);    }# k go now sub; grabs input from file ($username, $password) tries to loginsub k_go_now{    my ($username,          $password) = @_;          $ua = LWP::UserAgent->new;          $ua->agent("MyApp/0.1 ");          # Create a request          my $req = HTTP::Request->new(POST => 'http://search.cpan.org/search');          $req->content_type('application/x-www-form-urlencoded');          $req->content('query=libwww-perl&mode=dist');          # Pass request to the user agent and get a response back          my $res = $ua->request($req);          # Check the outcome of the response          if ($res->is_success) {                check_output ($res->content);          }           else {                print $res->status_line, "\n";            }}# suib check_output checks the response from the website and if it's any goodsub check_output{    my ($output) = @_;   if ($output =~ /Login Failed/)    {        print "Nope, failed login\n";   } elsif ($output =~ /Login Successful/)    {         print "got'em with login details\n";    } else    {        print "unknown response !!!\n$output\n\n";    }}

[sjxx]

Thank you,
What would I need to change if the form I needed to fill out were 25 digits in a single row?

[myth]

Thank you,
What would I need to change if the form I needed to fill out were 25 digits in a single row?

Huh ?

[sjxx]

For the https page it visits,
It needs to enter 25 digits
not a username/password

what part of the script would I need to change to achieve that?

[myth]

try and learn some scripting and programming it will help you out more then requesting.

I must agree with DiabloHorn, perl is a good language for scripting, especially doing exactly what your asking.

#!/usr/bin/perl -wuse LWP::UserAgent;my $digits;open( FILE, "< /tmp/blah.txt" ) or die "Can't open $filename : $!";while( <FILE> ) {$digits = $_;print "Attempting line: " . $digits;k_go_now($digits);# Sleep to keep us slightly more hidden and load off their serverssleep(1);}# k go now sub; grabs input from file ($username, $password) tries to loginsub k_go_now{my ($digits) = @_;$ua = LWP::UserAgent->new;$ua->agent("MyApp/0.1 ");# Create a requestmy $req = HTTP::Request->new(POST => 'http://search.cpan.org/search');$req->content_type('application/x-www-form-urlencoded');$req->content('query=libwww-perl&mode=dist');# Pass request to the user agent and get a response backmy $res = $ua->request($req);# Check the outcome of the responseif ($res->is_success) {check_output ($res->content);}else {print $res->status_line, "\n";}}# suib check_output checks the response from the website and if it's any goodsub check_output{my ($output) = @_;if ($output =~ /Login Failed/){print "Nope, failed login\n";} elsif ($output =~ /Login Successful/){print "got'em with login details\n";} else{print "unknown response !!!\n$output\n\n";}}

[sjxx]

Thanks man,
I'm pretty sure I got it working. Just one question, I tried googling the error but didn't get much luck.



Apperantly it can't reach the page or something,
Probably because it has to log in to an account to get to that page
It was my fault for assuming PERL would use cookies and auto login to the page, hehe
I think I know how to fix that though

[myth]

Apperantly it can't reach the page or something,
Probably because it has to log in to an account to get to that page
It was my fault for assuming PERL would use cookies and auto login to the page, hehe
I think I know how to fix that though

That shouldnt be difficult to figure out what its doing, you should just be able to find out where it has moved to (http://www.w3.org/Pr...2616-sec10.html), you can use something like telnet onto port 80 or wireshark to see where its telling you to go to, or simply remove the status 200 OK checking and let LWP attempt to work it out itself.

[sjxx]

ok,
i switched some things around so i don't have to deal with logging in,
it's your original script

#!/usr/bin/perl -wuse LWP::UserAgent;my $username;my $password;my $filename = '/tmp/blah.txt';open( FILE, "< /tmp/blah.txt" ) or die "Can't open $filename : $!"; while(  ) { print "Attempting line: " . $_; ($username, $password) = split (/ /, $_); k_go_now($username, $password); # Sleep to keep us slightly more hidden and load off their servers sleep(1); }# k go now sub; grabs input from file ($username, $password) tries to loginsub k_go_now{ my ($username, $password) = @_; $ua = LWP::UserAgent->new; $ua->agent("MyApp/0.1 "); # Create a request my $req = HTTP::Request->new(POST => 'https://steamcommunity.com/'); $req->content_type('application/x-www-form-urlencoded'); $req->content('query=libwww-perl&mode=dist'); # Pass request to the user agent and get a response back my $res = $ua->request($req); # Check the outcome of the response if ($res->is_success) { check_output ($res->content); } else { print $res->status_line, "\n"; }}# suib check_output checks the response from the website and if it's any goodsub check_output{ my ($output) = @_; if ($output =~ /Login Failed/) { print "Nope, failed login\n"; } elsif ($output =~ /Login Successful/) { print "got'em with login details\n"; } else { print "unknown response !!!\n$output\n\n"; }}

I'm not sure how it knows whether the login was succesful or not, at this point it just reads off the html in the page with every login it tries, I'm not sure how to tell it either.

I looked at this,
http://search.cpan.o...WP/UserAgent.pm

but it didn't help much

[myth]

# suib check_output checks the response from the website and if it's any goodsub check_output{ my ($output) = @_; if ($output =~ /Login Failed/) { print "Nope, failed login\n"; } elsif ($output =~ /Login Successful/) { print "got'em with login details\n"; } else { print "unknown response !!!\n$output\n\n"; }}

I'm not sure how it knows whether the login was succesful or not, at this point it just reads off the html in the page with every login it tries, I'm not sure how to tell it either.

Its in the sub check_output. The page SHOULD be returned in the string $output (try printing it). If its finds the text 'Login Successful' it assume the login was correct, if it finds 'Login Failed' it assumes the login failed, or if it finds anything else, it prints the output. Thats how it should work.. So change the next Login Successful to test you know will be displayed if the login was OK, and change Login Failed to something you know the page displays only when a login fails.

[sjxx]

Ok, I swapped the login successful / not successful to what it should say on the page, but everytime I run the script, after every login, it displays all the html for the page, doesn't exactly print the login succesful, or not

heres the one I am currently using,

[codebox]#!/usr/bin/perl -w



use LWP::UserAgent;



my $username;

my $password;

my $filename = '/tmp/blah.txt';



open( FILE, "< /tmp/blah.txt" ) or die "Can't open $filename : $!";

while( ) {

print "Attempting line: " . $_;

($username, $password) = split (/ /, $_);

k_go_now($username, $password);



# Sleep to keep us slightly more hidden and load off their servers

sleep(1);

}



# k go now sub; grabs input from file ($username, $password) tries to login

sub k_go_now

{

my ($username,

$password) = @_;

$ua = LWP::UserAgent->new;

$ua->agent("MyApp/0.1 ");



# Create a request

my $req = HTTP::Request->new(POST => 'https://steamcommunity.com/');

$req->content_type('application/x-www-form-urlencoded');

$req->content('query=libwww-perl&mode=dist');



# Pass request to the user agent and get a response back

my $res = $ua->request($req);



# Check the outcome of the response

if ($res->is_success) {

check_output ($res->content);

}

else {

print $res->status_line, "\n";

}

}



# suib check_output checks the response from the website and if it's any good

sub check_output

{

my ($output) = @_;





if ($output =~ /Incorrect login/)

{

print "Nope, failed login\n";

} elsif ($output =~ /Welcome/)

{

print "got'em with login details\n";

} else

{

print "unknown response !!!\n$output\n\n";

}

} [/codebox]


EDIT: Hmm, I think I got it working. I fiddled around with it and am getting back responses rather then unknown, I'll fiddle around with it a bit more =)

EDIT2: I went to the site, and when someone types in an incorrect username+password, under the login box it displays "Incorrect Login"

I put that into the script,

if ($output =~ /Incorrect Login/)
{
print "Nope, failed login\n";



but it doesn't recognize it for some reason

[sjxx]

got it working thanks

[myth]

Good to hear.. Perl is a good practical extraction and reporting language anyway