2007-11-27

WinHTTP magic

My task was to write a simple HTTP client. I tried URL monikers and WinHTTP API.

I started with URL monikers. Required library `urlmon.lib` comes with latest Microsoft Windows SDK, so you will probably want to install it first. You should know how to link libraries and include headers, so we will skip that step. In `urlmon.lib` we have a magic function `URLDownloadToFile()`, which can make your life really easy. If downloading the file into your home folder is everything you want, then use that function and feel happy.

But `URLDownloadToFile()` uses only `GET` method. I needed both `GET` and `POST`. Besides, creating a local copy of the file was unacceptable. Still avoiding sockets, I payed attention to WinHTTP API. Once again, it comes in `winhttp.lib` with Microsoft Windows SDK. There is a working example on related page in MSDN, you should copy-paste it, compile and run.

Works like a charm, doesn't it? And now you want to provide your own URL as a function parameter. And here it comes, the rocket science. First of all, you need to split the URL to get the host name, path and maybe some other data (getting host name and path were sufficient for me). I googled a bit and found `WinHttpCrackUrl()` function.

The first surprise is that it accepts URLs of type `LPCWSTR` (long pointer to constant wide string), not `char *`. In case you need to convert `char *` to `LPCWSTR`, google for soultion or use this one. It is not mine, I just tweaked it a bit:

// Gets URL (converted to widechar string) length
local_url_len = MultiByteToWideChar(CP_UTF8, 0, url, -1, NULL, 0);
// Allocates memory
local_url = malloc(sizeof(WCHAR) * local_url_len);
// Converts URL to widechar string
MultiByteToWideChar(CP_UTF8, 0, url, -1, local_url, local_url_len);


Then we send `local_url` to `WinHttpCrackUrl()` (see also MSDN example):

WinHttpCrackUrl(local_url, wcslen(local_url), 0, &urlComp);


If you think that you can now use `urlComp` structure members like `urlComp.lpszHostName` and `urlComp.lpszUrlPath` to connect wherever you want, you are totally wrong. You will get problems with the host name. Try this:

real_host_name = malloc(sizeof(WCHAR) * urlComp.dwHostNameLength);
wcsncpy_s(real_host_name, urlComp.dwHostNameLength + 1, urlComp.lpszHostName, _TRUNCATE);

wprintf(L"DEBUG: %d | %d | %s\n",
wcslen(real_host_name), urlComp.dwHostNameLength, real_host_name);
wprintf(L"DEBUG: %d | %d | %s\n",
wcslen(urlComp.lpszHostName), urlComp.dwHostNameLength, urlComp.lpszHostName);
wprintf(L"DEBUG: %d | %d | %s\n",
wcslen(urlComp.lpszUrlPath), urlComp.dwUrlPathLength, urlComp.lpszUrlPath);


The fact is, `urlComp.lpszHostName` includes URL path part as well! Why? Hell knows, really. Although `urlComp.dwHostNameLength` contains the right size of host name (in wide chars), it is not used by `WinHttpConnect()`. For `http://www.domain.com/service` you will get host name length equal to 14, which is correct, but host name equal to `www.domain.name.com/service`, which is wrong. As a result, you will probably get an error trying to connect like that:

// Will probably give error while executing
hConnect = WinHttpConnect(hSession, urlComp.lpszHostName, INTERNET_DEFAULT_HTTP_PORT, 0);


Try this instead:

hConnect = WinHttpConnect(hSession,
real_host_name, INTERNET_DEFAULT_HTTP_PORT, 0);

hRequest = WinHttpOpenRequest(hConnect, NULL, urlComp.lpszUrlPath, 0,
WINHTTP_NO_REFERER, WINHTTP_DEFAULT_ACCEPT_TYPES, 0);


Hope that saved your time. Because nobody seemed to worry about mine.

1 comment:

Anonymous said...

Thanks a lot.