Harriet Bazley
2022-01-21 13:43:58 UTC
I've been trying to use wget to retrieve a page that is only accessible
to logged-in users (user stats - so that I can analyse them and keep a
running record of changes).
Basically, I can't seem to get the correct syntax for the site to
receive/recognise my name and password in the first place, let alone to
serve up the stats page requested....
I don't really know how to use the relevant features of
wget and have been flailing around rather at random. Simply using
wget --ask-password STATS_URL
doesn't produce the desired result; it prompts for the password all
right, but when I supply it the fetch then gets redirected to retrieve
the log-in page instead, just as if I had supplied no password or the
wrong one.
wget --user=USERNAME --ask-password STATS_URL
prompts "Password for user" instead of just "Password", but still
doesn't seem to pass the required data.
Same result from
wget --user=USERNAME --password=PASS STATS_URL
(the retrieved page states 'sorry, you don't have access to view the
page you were trying to reach, please log in')
After looking for advice on the Web I tried fetching the log-in page
directly using the same methods and using --keep-session-cookies before
running a second command to fetch the stats page immediately afterwards,
but that didn't work. It fetches the login page, then redirects and
fetches it again under a different name, the only difference being the
error:
<div class="flash error">Sorry, you don't have permission to access the page you were trying to reach. Please log in.</div>
I then tried using --save-cookies followed by --load-cookies for the
second request, but that didn't work, doubtless because the resulting
'cookies' file had no content:
# HTTP cookie file.
# Generated by Wget on 2022-01-21 13:37:33.
# Edit at your own risk.
I then tried
wget --post-data 'user_login=USERNAME&user_password=PASS' LOGIN_URL
where the relevant form reads
<dt><label for="user_login">User name or email:</label></dt>
<dd><input type="text" name="user[login]" id="user_login"/></dd>
<dt><label for="user_password">Password:</label></dt>
<dd><input type="password" name="user[password]" id="user_password"/></dd>
<dt><label for="user_remember_me">Remember me</label></dt>
<dd><input name="user[remember_me]" type="hidden" value="0"/><input type="checkbox" value="1" name="user[remember_me]" id="user_remember_me"/></dd>
<dt class="landmark">Submit</dt>
<dd class="submit actions">
<input type="submit" name="commit" value="Log in" class="submit"/>
</dd>
but still had no luck.
I'm simply not managing to submit the name/password combination in any
way that the site will acknowledge.
to logged-in users (user stats - so that I can analyse them and keep a
running record of changes).
Basically, I can't seem to get the correct syntax for the site to
receive/recognise my name and password in the first place, let alone to
serve up the stats page requested....
I don't really know how to use the relevant features of
wget and have been flailing around rather at random. Simply using
wget --ask-password STATS_URL
doesn't produce the desired result; it prompts for the password all
right, but when I supply it the fetch then gets redirected to retrieve
the log-in page instead, just as if I had supplied no password or the
wrong one.
wget --user=USERNAME --ask-password STATS_URL
prompts "Password for user" instead of just "Password", but still
doesn't seem to pass the required data.
Same result from
wget --user=USERNAME --password=PASS STATS_URL
(the retrieved page states 'sorry, you don't have access to view the
page you were trying to reach, please log in')
After looking for advice on the Web I tried fetching the log-in page
directly using the same methods and using --keep-session-cookies before
running a second command to fetch the stats page immediately afterwards,
but that didn't work. It fetches the login page, then redirects and
fetches it again under a different name, the only difference being the
error:
<div class="flash error">Sorry, you don't have permission to access the page you were trying to reach. Please log in.</div>
I then tried using --save-cookies followed by --load-cookies for the
second request, but that didn't work, doubtless because the resulting
'cookies' file had no content:
# HTTP cookie file.
# Generated by Wget on 2022-01-21 13:37:33.
# Edit at your own risk.
I then tried
wget --post-data 'user_login=USERNAME&user_password=PASS' LOGIN_URL
where the relevant form reads
<dt><label for="user_login">User name or email:</label></dt>
<dd><input type="text" name="user[login]" id="user_login"/></dd>
<dt><label for="user_password">Password:</label></dt>
<dd><input type="password" name="user[password]" id="user_password"/></dd>
<dt><label for="user_remember_me">Remember me</label></dt>
<dd><input name="user[remember_me]" type="hidden" value="0"/><input type="checkbox" value="1" name="user[remember_me]" id="user_remember_me"/></dd>
<dt class="landmark">Submit</dt>
<dd class="submit actions">
<input type="submit" name="commit" value="Log in" class="submit"/>
</dd>
but still had no luck.
I'm simply not managing to submit the name/password combination in any
way that the site will acknowledge.
--
Harriet Bazley == Loyaulte me lie ==
We are not punished for our sins, but by them.
Harriet Bazley == Loyaulte me lie ==
We are not punished for our sins, but by them.