Re: [WM]: ROC curve!

[Elias <elhorzo@aiia.csd.auth.gr>]

=CB=EF =B3=A4=B8=BB wrote:
> I am trying to create ROC curves for a watermarking scheme I am =
testing.
> However, I am unsure how to go about calculating the probability of
> false detection and the probability of false rejection in order to =
plot
> the ROC curve. Can you give me any information on how to go about
> plotting the cuve I would appreciate it.
> Thanks in advance!
>
>

The probability of false detection (P_fd) consist in detecting a
non-watermarked object as it is watermarked. Also it is to watermark an
object using key 1 (K1) and (wrongly) detect it as if it was watermarked
using key 2 (K2), where K2 is different to K1.

The probability of false rejection (P_fr) consist in not detecting an
object that was watermarked using a key 1 using the same key K1 in the
detection stage.

The number of experiment you need to perform depends on the
approximately value of the probability you are expecting. In simulations
a usual practice is to set N =3D 10 / Prob. For instance, if you expect =
a
probability around 10^-5 then you should use N>=3D 10^6.
To determine N you can also use the Chernoff bound:
   N >=3D (1/2 * err^2) * ln(2/s)
So, after performing N experiments, you have a confidence of at least
 (1 - 2*exp(-2 * N * err^2))
that the difference between the real probability and the empirical
probability is no more than 'err'

Therefore, for computing P_fd and P_fr:


fd =3D 0;
fr =3D 0;
N =3D number of experiments;
for i =3D 1:N
  0. Generate two different keys K1 and K2
  1. watermark an object using key K1
  2. attack the watermarked object
  3. Use K1 to detect if the object is watermarked
        if you DO NOT detect it as watermarked then fr =3D fr+1;
  4. Use K2 to detect if the object is watermarked
        if you DO detect it as watermarked then fd =3D fd+1;
end

P_fd =3D fd / N;
P_fr =3D fr / N;

The step 2, the attacks, it is only if you are evaluating the
performance under attacks, if you are not then, obviously, don't attack
the watermarked object.

So far you only have a pair of values {P_fd, P_fr}, now we'll focus on
computing a series of P_fd and P_frof in order to calculate the ROC =
curve.

Let's suppose you decide if an object has been or not watermarked
comparing the output of a function against a threshold, like

detect(object, key1) =3D d, where d is a real value in [0,1]

if the threshold =3D th, where th is a real value in [0,1], then

if d >=3D th --> the object has been watermarked using key1
if d <  th --> the object has not been watermarked using key1

So first you build the output vectors

N =3D number of experiments;
outputs_k1 =3D zeros(N,1);
outputs_k2 =3D zeros(N,1);
for i =3D 1:N
  0. Generate two different keys K1 and K2
  1. watermarked_object =3D watermark_function(object,K1)
  2. watermarked_object =3D apply_attacks(watermarked_object)
  3. outputs_k1[i] =3D detect(watermarked_object, K1)
  4. outputs_k2[i] =3D detect(watermarked_object, K2)
end

Again the attacks is optional.

And finally, for evaluating the ROC curve, you can use something similar
to this simple Matlab script.

function [falsenegative,falsepossitive] =3D roc(outputs_k1, outputs_k2)
% function [falsenegative,falsepossitive] =3D roc(outputs_k1, =
outputs_k2)
%
% INPUTS:
% outputs_k1 =3D the outputs of the detection function using the correct =
key,
%              that is, the key it was used for watermarking
% outputs_k2 =3D the outputs of the detection function using an =
erroneous
key,
%              that is, a different key from the one which was used for
watermarking
%
% the values of outputs_k1 and outputs_k2 must be in [0,1] and both =
vectors
% must have the same length
%
% OUTPUTS:
% falsenegative  =3D Probability of false rejection
% falsepossitive =3D Probability of false detection

outputs_k1 =3D outputs_k1(:);
outputs_k2 =3D outputs_k2(:);

thres =3D 0:0.001:1;

N =3D length(outputs_k1);

for(k=3D1:length(thres))
    falsenegative(k)  =3D sum(outputs_k1<thres(k));
    falsepossitive(k) =3D sum(outputs_k2>=3Dthres(k));
end

falsenegative  =3D falsenegative  / N;
falsepossitive =3D falsepossitive / N;

figure,
plot(falsenegative,falsepossitive)
set(gca,'XScale','log')
set(gca,'YScale','log')
title('ROC')
xlabel('False rejection probability');
ylabel('False alarm probability');


Good luck.





______________________________________________________________________________

Watermarking Mailing List - http://www.watermarkingworld.org/ml.html
To unsubscribe send email to "majordomo@watermarkingworld.org" with
"unsubscribe watermarking YOURMAIL" in the body.
______________________________________________________________________________