How can I safely use $_SERVER to avoid XSS?

I’ve seen a lot of warnings about using $_SERVER['REQUEST_URI'] because it can open things up to XSS, but I haven’t been able to find anything confirming a safe way to use it. Some people have mentioned using esc_url(), but I wasn’t able to find anything confirming how to safely use it. This is my best guess, would this safely prevent XSS attacks?

echo esc_url(( is_ssl() ? 'https://' : 'http://' ) . $_SERVER['HTTP_HOST'] . $_SERVER['REQUEST_URI']);

If I want to use the URL internally rather than display it, it seems I have to use esc_url_raw(). Is this safe as well?

$pageurl = esc_url_raw(( is_ssl() ? 'https://' : 'http://' ) . $_SERVER['HTTP_HOST'] . $_SERVER['REQUEST_URI']);

Solutions Collecting From Web of "How can I safely use $_SERVER to avoid XSS?"