It is natural to find varnish public ip in apache logs but not very secure in my context, so I'll need to configure Varnish 3 using req.http.X-Forwarded-For and apache2 with mod_rpaf

With a debian like:

On the backend server

apt-get install libapache2-mod-rpaf
nano /etc/apache2/mods-enabled/rpaf.conf
<IfModule rpaf_module>
    RPAFenable On

    # When enabled, take the incoming X-Host header and
    # update the virtualhost settings accordingly:
    RPAFsethostname On

    # Define which IP's are your frontend proxies that sends
    # the correct X-Forwarded-For headers:
    RPAFproxy_ips ::1

    # Change the header name to parse from the default
    # X-Forwarded-For to something of your choice:
    RPAFheader X-Forwarded-For

service apache2 reload

On the reverse proxy / varnish server:

Insert after the start of sub vcl_recv :

if (req.restarts == 0) {
  if (req.http.X-Forwarded-For) {
    set req.http.X-Forwarded-For = req.http.X-Forwarded-For + ", " + client.ip;
  } else {
    set req.http.X-Forwarded-For = client.ip;

Then reload the configuration and verify a2 logs.