Description of parameters in evaluation_config.yaml

The evaluation_config.yaml file contains the necessary parameters for running the Scissorshands evaluation framework. Below is a detailed description of each parameter along with examples.

Model Configuration:

  • ckpt_path : Path to the finetuned Stable Diffusion checkpoint file to be evaluated.
  • Type: str

  • Example: "outputs/scissorshands/finetuned_models/scissorshands_Abstractionism_model.pth"

  • classification_model : Specifies the classification model used for evaluating the generated outputs.

  • Type: str
  • Example: "vit_large_patch16_224"

Training and Sampling Parameters:

  • devices : CUDA device IDs to be used for the evaluation process.
  • Type: str

  • Example: "0"

  • cfg_text : Classifier-free guidance scale value for image generation. Higher values increase the strength of the conditioning prompt.

  • Type: float

  • Example: 9.0

  • seed : Random seed for reproducibility of results.

  • Type: int

  • Example: 188

  • ddim_steps : Number of steps for the DDIM (Denoising Diffusion Implicit Models) sampling process.

  • Type: int

  • Example: 100

  • ddim_eta : DDIM eta value for controlling the amount of randomness during sampling. Set to 0 for deterministic sampling.

  • Type: float

  • Example: 0.0

  • image_height : Height of the generated images in pixels.

  • Type: int

  • Example: 512

  • image_width : Width of the generated images in pixels.

  • Type: int
  • Example: 512

Output and Logging Parameters:

  • sampler_output_dir : Directory where generated images will be saved during evaluation.
  • Type: str

  • Example: "outputs/eval_results/mu_results/scissorshands/"

  • eval_output_dir : Directory where evaluation metrics and results will be stored.

  • Type: str

  • Example: "outputs/eval_results/mu_results/scissorshands/"

  • reference_dir : Directory containing original images for comparison during evaluation.

  • Type: str
  • Example: "msu_unlearningalgorithm/data/quick-canvas-dataset/sample/"

Performance and Efficiency Parameters:

  • multiprocessing : Enables multiprocessing for faster evaluation for FID score. Recommended for large datasets.
  • Type: bool

  • Example: False

  • batch_size : Batch size used during FID computation and evaluation.

  • Type: int
  • Example: 16

Optimization Parameters:

  • forget_theme : Concept or style intended for removal in the evaluation process.
  • Type: str

  • Example: "Bricks"

  • seed_list : List of random seeds for performing multiple evaluations with different randomness levels.

  • Type: list

  • Example: ["188"]

  • use_sample: If you want to just run on sample dataset then set it as True. By default it is True.

  • Type: bool
  • Example: True