3.1 Data Loading and Initialization

• Read and import the lab measurement data from the CSV file.

  df_lab = pd.read_csv("LabMeasurements-Color-Card.csv",sep=';', decimal=',') 

• Initialize the dictionary to store data for each sheet.

  sheets = {}  

3.2 Sheet Data Extraction and Concatenation

• Iterate through each sheet to extract relevant data based on rows and columns.
• Concatenate selected data into a comprehensive DataFrame for each sheet.

  new_df = pd.concat(sheet_1, axis=1).T.reset_index(drop=True)

3.3 Reshape Data for Analysis

• Extracting and reshaping to prepare for structured storage.

  
                                      for index, row in test_df.iterrows():
                                          # Iterate over each 8x8 block in the row
                                          for i in range(1, 9):
                                              for j in range(1, 9):
                                                  # Extract the L, a, b values for the block
                                                  L = row[f'L{i}{j}']
                                                  a = row[f'a{i}{j}']
                                                  b = row[f'b{i}{j}']
  
                                                  # Append the values to the reshaped data
                                                  reshaped_data.append({
                                                      'row': initial_row,
                                                      'column': initial_column,
                                                      'L': L,
                                                      'a': a,
                                                      'b': b
                                                  })

• Creating reshaped_df to organize color card data into a structured format.

                                    reshaped_df = reshaped_df.astype({
                                        'row': int,
                                        'column': int,
                                        'L': float,
                                        'a': float,
                                        'b': float
                                    })
                

3.4 Create Nested Dictionaries

• In this step we have created the dictionary of dictionaries where each key of the parent dictionary is the sheet number, and the value is the dictionary of that particular sheet , for the child dictionary, key is the 'card'number, value is the 'L,a,b' dataframe.
• Next we have populated dictionaries within each sheet to store color card data.

sheets[f'{no_sheets}'] = list_of_cards

3.5 Data_Processing_Workflow

5.1 Find Variance for Each Trial/Sheet

This function, "calculate_variance_for_sheet", helps us understand the diversity of colors across a collection of color cards. Imagine each color card as a collection of numbers that describe its color: how bright it is (L), how much red or green it has (a), and how much blue or yellow it has (b). This function looks at all these numbers from all the color cards in a sheet, calculates how much these numbers vary from card to card, and then gives us a single number that tells us, on average, how much the colors differ across all the cards. This helps in understanding if the colors in the group are mostly similar or if there's a wide range of different colors present. It's like measuring 'how consistent or varied the colors are in a set of card in the particular sheet.

    def calculate_variance_for_sheet(sheet):
    lab_values = []
    for card_df in sheet.values():
        lab_values.append(card_df[['L', 'a', 'b']].values)  # Extract L*a*b* values from DataFrame

    lab_values = np.concatenate(lab_values, axis=0)
    return np.var(lab_values, axis=0).mean()
    

5.2 Find the Sheet with minimum Variance

It calculates the variance of color properties for each sheet, finds the sheet with the minimum variance, and then prints its identifier (min_variance_sheet_id) along with variance values for all sheets. Finally, it retrieves the color cards from this sheet (min_variance_sheet) for further examination or presentation. This helps in determining which group of color cards shows the most consistent colors across the dataset. As a result we find the sheet Number 6 is the most consistent trial amongs all the 13 trials

    min_variance_sheet_id = min(variances, key=variances.get)

    print(f"Sheet with minimum variance is: {min_variance_sheet_id}")
    print(f"Variance values for each sheet: {variances}")

    # Accessing the sheet with minimum variance
    min_variance_sheet = sheets[min_variance_sheet_id]

Output

    Sheet with minimum variance is: 5
    Variance values for each sheet: {'0': 313.98876421195024, '1': 311.23875731943366, '2': 314.87699874555045, '3': 315.02180117684867, '4': 308.6178636086258, '5': 303.3818061495981, '6': 306.7759366134873, '7': 310.5400739589045, '8': 318.49616522486843, '9': 309.8834551776406, '10': 309.16972715804945, '11': 310.8522839363206, '12': 307.5054587806306}
    

5.3 Normalised Efficiency Distribution of 42 Cards from Sheet 6 with MasterCard

The Intution Behind Normalised Efficiency Distribution :

After we have found the most consistent trial sheet, we want to visualise how the each 42 cards was close to the MasterCard, in other words, How the efficiency of the 42 cards distributed so that we can haveidea of Trial Progession in the SHeet Number 6. It is worth to note, that as the trial approach to the end, the chance of error diminish for the colors near to the Gap, but this is not strongly consistent for the colors on the far edge.

for i in range(8):
    for j in range(8):
        delta_e_values = []
        color_block_master = master_card.iloc[8*i + j]
        for color_card in least_variant_sheet.values():
            color_block_other = color_card.iloc[8*i + j]
            L_master, a_master, b_master = color_block_master['L'], color_block_master['a'], color_block_master['b']
            L_other, a_other, b_other = color_block_other['L'], color_block_other['a'], color_block_other['b']
            delta_e_values.append(calculate_delta_e(L_master, a_master, b_master, L_other, a_other, b_other))

Data Normalisation

    # Normalize ΔE values within each square
    normalized_delta_e_values = (delta_e_values - np.min(delta_e_values)) / (np.max(delta_e_values) - np.min(delta_e_values))
                        

Visualisation

5.4 Heatmap Visualisation of Accuracy of trials in Sheet No. 6

The Purpose of Heatmap Visualisation to find acurracies of each 42 cards

This part compares color differences between a master color card and multiple trial color cards in a specific sheet. It calculates the differences for each color block in an 8x8 grid using the Delta E formula, which measures the Euclidean distance between color values. Each trial card's differences are visualized as heatmaps, highlighting the accuracy of the trials compared to the master card. The heatmaps for all cards in the sheet are plotted together, with a color bar indicating the magnitude of color differences, providing a clear visual representation of how closely each trial card matches the master card.

Again it is worth to note that the sqaure at the position (6,2) has largest del E, these signify that this is square is the significantly of different color than MasterCard's square at the same position.

Code

    def plot_heatmaps_for_sheet(sheet, master_card):
    fig, axes = plt.subplots(6, 7, figsize=(21, 18))
    axes = axes.flatten()
    max_delta_e = 0
    for idx, card_id in enumerate(sheet):
        delta_e_matrix = np.zeros((8, 8))
        color_card = sheet[card_id]
        for i in range(8):
            for j in range(8):
                color_block_master = master_card.iloc[8*i + j]
                color_block_card = color_card.iloc[8*i + j]
                L_before, a_before, b_before = color_block_master['L'], color_block_master['a'], color_block_master['b']
                L_after, a_after, b_after = color_block_card['L'], color_block_card['a'], color_block_card['b']
                delta_e_matrix[i, j] = calculate_delta_e(L_before, a_before, b_before, L_after, a_after, b_after)
                if delta_e_matrix[i, j] > max_delta_e:
                    max_delta_e = delta_e_matrix[i, j]
        sns.heatmap(delta_e_matrix, cmap='hot', cbar=False, ax=axes[idx], annot=False, vmin=0, vmax=max_delta_e)
        axes[idx].invert_yaxis()  # Invert the y-axis

        axes[idx].set_title(f'Card {card_id}')
        axes[idx].axis('off')

    # Add a colorbar below the plot
    cbar_ax = fig.add_axes([0.3, 0.02, 0.4, 0.02])  # Position: left, bottom, width, height
    norm = plt.Normalize(vmin=0, vmax=max_delta_e)
    sm = plt.cm.ScalarMappable(cmap='hot', norm=norm)
    sm.set_array([])
    fig.colorbar(sm, cax=cbar_ax, orientation='horizontal', label='Color Difference (ΔE)')
    plt.suptitle("Accuracy of trials in Sheet No. 6", fontsize=25, y=1)
    plt.tight_layout(rect=[0, 0.05, 1, 1])
    plt.show()

Heatmap Visualisation of Acurracy

6.1. Step - 1 : Calculate the Mean of Sheet-6

This step calculates the average color values (L*, a*, and b*) for each of the 64 color blocks across all color cards in a sheet. It iterates through each card in the sheet, extracting the L*, a*, and b* values for all blocks and appending these values to the respective lists. Once all values are collected, the function stacks these lists horizontally, effectively organizing all L*, a*, and b* values for each block into columns. It then computes the average of these values for each block. Finally, the function creates a new table (DataFrame) containing the average L*, a*, and b* values for each of the 64 color blocks and returns this table, providing a comprehensive overview of the average color properties for the sheet.

Code

    # Function to calculate mean L*, a*, b* values for each of the 64 rows across all cards in a sheet
    def calculate_mean_for_sheet(sheet):
        # Initialize lists to store the L, a, b values for each row
        L_values = []
        a_values = []
        b_values = []

        # Loop through each card in the sheet and extract the L, a, b columns
        for card in sheet.values():
            L_values.append(card['L'].values)
            a_values.append(card['a'].values)
            b_values.append(card['b'].values)

        # Stack the values horizontally and calculate the mean for each row
        L_values = np.stack(L_values, axis=1)
        a_values = np.stack(a_values, axis=1)
        b_values = np.stack(b_values, axis=1)

        L_mean = np.mean(L_values, axis=1)
        a_mean = np.mean(a_values, axis=1)
        b_mean = np.mean(b_values, axis=1)

        # Create a DataFrame to store the mean values
        mean_df = pd.DataFrame({
            'L': L_mean,
            'a': a_mean,
            'b': b_mean
        })

        return mean_df

6.2. Step - 2 : Find the Card closest to the Mean of Sheet-6

This stepidentifies the color card that is closest to the average color values in sheet '6'. It iterates through each card in the sheet and calculates the distance between the card's color values and the average color values for the sheet. This distance is measured using a pre-defined method. During this process, the code keeps track of the card that has the smallest distance to the average. After comparing all the cards, the code determines which card is closest to the mean color values and outputs the identifier of this card along with the distance. This helps in identifying the card that best represents the average color properties of the sheet.

Code

for card_id, card in sheets['5'].items():
  # Calculate the distance to the mean DataFrame
  distance = calculate_distance(card[['L', 'a', 'b']], mean_df_sheet_5)

  # Update the closest card if the current distance is smaller
  if distance < min_distance:
      min_distance = distance
      closest_card = card_id

print(f"The card closest to the mean in sheet '6' is card '{closest_card + 1}' with a distance of {min_distance:.4f}")

Output

    The card closest to the mean in sheet '6' is card '23' with a distance of 0.4511

6.3 Visualise the Card 23 from Sheet 6 || The Most likely to Be Mean from the most Consistent Sheet

Let's see how our select card looks like ..

A very spontaneous difference can be observed, that is the color of the Square (6,2) of the card 23 is very different from the MasterCard's one

6.4 Visualise Circular Difference between the Card 23 from Sheet 6 with MasterCard

The intution behind plot

In the following plot, del E is being calculated for the Card 23 from Sheet 6 with MasterCard. Now as an observer may be intersted to know huw intensely, each color square differ from each other, so for that intensity of delta E has been calculated and visualised in the rectangle, background of the Overlapped Circular Blocks, where left one is the from card 23 from Sheet 6 and right one is the MasterCard. This gives an idea of the most accurate color square out of 64 squares.

Explanation

The plot visualizes the color differences between "Card 23" from "Sheet 6" and the reference "MasterCard." It uses background intensity and color overlays to highlight discrepancies. The intensity of each cell's background is based on the color difference, calculated using Euclidean distance in the L*, a*, b* color space, and normalized to a maximum value of 100. This grayscale background, where darker shades indicate higher color differences, provides a clear indication of color variance. Overlaid on each cell are two circles representing the actual colors from the ideal card and the MasterCard, with colors converted from L*, a*, b* values to RGB for accurate visualization. The gridlines enhance the clarity of individual sections, and a color bar labeled "Color Difference Intensity" offers a reference scale. The Y-axis is inverted for standard visual orientation. This combined approach of intensity mapping and color overlays offers a comprehensive view of color discrepancies, crucial for quality control in color-critical applications. The plot effectively identifies areas of significant color variance, aiding in a detailed analysis of color accuracy between the cards.

6.3 Closest Cards to Mean Visualization