Tag Archives: precision

Dr. Ed Askew
From The Lab

Quality Plans for Lab Services: Managing Risks as a Grower, Processor or Dispensary, Part 3

By Dr. Edward F. Askew
No Comments
Dr. Ed Askew

Editor’s Note: The views expressed in this article are the author’s opinions based on his experience working in the laboratory industry. This is an opinion piece in a series of articles designed to highlight the potential problems that clients may run into with labs. 


In the last two articles, I discussed the laboratory’s first line of defense (e.g. certification or accreditation) paperwork wall used if a grower, processor or dispensary (user/client) questioned a laboratory result and the conflicts of interest that exist in laboratory culture. Now I will discuss the second line of defense that a laboratory will present to the user in the paperwork wall: Quality Control (QC) results.

Do not be discouraged by the analytical jargon of the next few articles. I suggest that you go immediately to the conclusions to get the meat of this article and then read the rest of it to set you on the path to see the forest for the trees.

QC in a laboratory consists of a series of samples run by the laboratory to determine the accuracy and precision of a specific batch of samples. So, to start off, let’s look at the definitions of accuracy and precision.QC Charts can provide a detailed overview of laboratory performance in a well-run laboratory.

According to the Standard Methods for the Examination of Water and Wastewater:

Accuracy: estimate of how close a measured value is to the true value; includes expressions for bias and precision.

Precision: a measure of the degree of agreement among replicate analyses of a sample.

A reputable laboratory will measure the Accuracy and Precision of QC samples in a batch of user samples and record these values in both the analytical test report issued to the user and in control charts kept by the laboratory. These control charts can be reviewed by the user if they are requested by the user. These control charts record:

Accuracy (means) chart: The accuracy chart for QC samples (e.g., LRB, CCV, LFBs, LFMs, and surrogates) is constructed from the average and standard deviation of a specified number of measurements of the analyte of interest.

Precision (range) chart: The precision chart also is constructed from the average and standard deviation of a specified number of measurements (e.g., %RSD or RPD) for replicate of duplicate analyses of the analyte of interest.

Now, let’s look at what should be run in a sample batch for cannabis analyses. The typical cannabis sample would have analyses for cannabinoids, terpenes, microbiological, organic compounds, pesticides and heavy metals.

Each compound listed above would require a specific validated analytical method for the type of matrix being analyzed. Examples of specific matrixes are:

  • Cannabis buds, leaves, oil
  • Edibles, such as Chocolates, Baked Goods, Gummies, Candies and Lozenges, etc.
  • Vaping liquids
  • Tinctures
  • Topicals, such as lotions, creams, etc.

Running QC analyses does not guarantee that the user’s specific sample in the batch was analyzed correctly.

Also, both ISO 17025-2005 and ISO 17025-2017 require the use of a validated method.

ISO 17025-2005: When it is necessary to use methods not covered by standard methods, these shall be subject to agreement with the customer and shall include a clear specification of the customer’s requirements and the purpose of the test and/or calibration. The method developed shall have been validated appropriately before use.

ISO 17025-2017: The laboratory shall validate non-standard methods, laboratory-developed methods and standard methods used outside their intended scope or otherwise modified. The validation shall be as extensive as is necessary to meet the needs of the given application or field of application.

Validation procedures can be found in a diverse number of analytical chemistry associations (such as AOACand ASTM) but the State of California has directed users and laboratories to the FDA manual “Guidelines for the Validation of Chemical Methods for the FDA FVM Program, 2nd Edition, 2015

The laboratory must have on file for user review the following minimum results in an analytical statistical report validating their method:

  • accuracy,
  • limit of quantitation,
  • ruggedness,
  • precision,The user must look beyond the QC data provided in their analytical report or laboratory control charts.
  • linearity (or other calibration model),
  • confirmation of identity
  • selectivity,
  • range,
  • spike recovery.
  • limit of detection,
  • measurement uncertainty,

The interpretation of an analytical statistical report will be discussed in detail in the next article. Once the validated method has been selected for the specific matrix, then a sample batch is prepared for analysis.

Sample Batch: A sample batch is defined as a minimum of one (1) to a maximum of twenty (20) analytical samples run during a normal analyst’s daily shift. A LRB, LFB, LFM, LFMD, and CCV will be run with each sample batch. Failure of any QC sample in sample batch will require a corrective action and may require the sample batch to be reanalyzed. The definitions of the specific QC samples are described later.

The typical sample batch would be set as:

  • Instrument Start Up
  • Calibration zero
  • Calibration Standards, Quadratic
  • LRB
  • LFB
  • Sample used for LFM/LFMD
  • LFM
  • LFMD
  • Samples (First half of batch)
  • CCV
  • Samples (Second half of batch)
  • CCV

The QC samples are defined as:

Calibration Blank: A volume of reagent water acidified with the same acid matrix as in the calibration standards. The calibration blank is a zero standard and is used to calibrate the ammonia analyzer

Continuing Calibration Verification (CCV): A calibration standard, which is analyzed periodically to verify the accuracy of the existing calibration for those analytes.

Calibration Standard: A solution prepared from the dilution of stock standard solutions. These solutions are used to calibrate the instrument response with respect to analyte concentration

Laboratory Fortified Blank (LFB): An aliquot of reagent water or other blank matrix to which known quantities of the method analytes and all the preservation compounds are added. The LFB is processed and analyzed exactly like a sample, and its purpose is to determine whether the methodology is in control, and whether the laboratory is capable of making accurate and precise measurements.

Laboratory Fortified Sample Matrix/Duplicate (LFM/LFMD) also called Matrix Spike/Matrix Spike Duplicate (MS/MSD): An aliquot of an environmental sample to which known quantities of ammonia is added in the laboratory. The LFM is analyzed exactly like a sample, and its purpose is to determine whether the sample matrix contributes bias to the analytical results. The background concentrations of the analytes in the sample matrix must be determined in a separate aliquot and the measured values in the LFM corrected for background concentrations (Section 9.1.3).Laboratories must validate their methods.

Laboratory Reagent Blank (LRB): A volume of reagent water or other blank matrix that is processed exactly as a sample including exposure to all glassware, equipment, solvents and reagents, sample preservatives, surrogates and internal standards that are used in the extraction and analysis batches. The LRB is used to determine if the method analytes or other interferences are present in the laboratory environment, the reagents, or the apparatus.

Once a sample batch is completed, then some of the QC results are provided in the user’s analytical report and all of the QC results should be recorded in the control charts identified in the accuracy and precision section above.

But having created a batch and performing QC sample analyses, the validity of the user’s analytical results is still not guaranteed. Key conclusion points to consider are:

  1. Laboratories must validate their methods.
  2. Running QC analyses does not guarantee that the user’s specific sample in the batch was analyzed correctly.
  3. QC Charts can provide a detailed overview of laboratory performance in a well-run laboratory.

The user must look beyond the QC data provided in their analytical report or laboratory control charts. Areas to look at will be covered in the next few articles in this series.

Ask The Expert: Exploring Cannabis Laboratory Accreditation Part 4

By Aaron G. Biros
No Comments

In the first part of this series, we spoke with Michelle Bradac, senior accreditation officer at A2LA, to learn the basics of cannabis laboratory accreditation. In the second part, we sat down with Roger Brauninger, A2LA Biosafety Program manager, to learn why states are looking to lab accreditation in their regulations for the cannabis industry. In the third part, we heard from Michael DeGregorio, chief executive officer of Konocti Analytics, Inc., discussing method development in the cannabis testing industry and his experience with getting accredited.

In the fourth and final part of this series, we sit down with Susan Audino, Ph.D., an A2LA lead assessor and instructor, laboratory consultant and board member for the Center for Research on Environmental Medicine in Maryland. Dr. Audino will share some insights into method validation and the most technical aspects of laboratory accreditation.

Susan Audino, Ph.D.

Susan Audino obtained her Ph.D. in Chemistry with an analytical chemistry major, physical and biochemistry minor areas. She currently owns and operates a consulting firm to service chemical and biological laboratories. Susan has been studying the chemistry and applications of cannabinoids and provides scientific and technical guidance to cannabis dispensaries, testing laboratories and medical personnel. Dr. Audino’s interest most directly involves cannabis consumer safety and protection, and promotes active research towards the development of official test methods specifically for the cannabis industry, and to advocate appropriate clinical research. In addition to serving on Expert Review Panels, she is also chairing the first Cannabis Advisory Panel and working group with AOAC International, is a member of the Executive Committee of the ASTM Cannabis Section and has consulted to numerous cannabis laboratories and state regulatory bodies.

CannabisIndustryJournal: What are the some of the most significant technical issues facing an accreditation body when assessing a cannabis-testing laboratory?

Susan: From the AB perspective, there needs to be a high level of expertise to evaluate the merits and scientific soundness of laboratory-developed analytical test methods. Because there are presently no standard or consensus test methods available, laboratories are required to develop their own methods, which need to be valid. Validating methods require a rigorous series of tests and statistical analyses to ensure the correctness and reliability of the laboratory’s product, which is– the test report.

CIJ: When is method validation required and how does this differ from system suitability?

Susan: Method validation is required whenever the laboratory modifies a currently accepted consensus or standard test method, or when the laboratory develops its own method. Method validation is characterized by a series of analytical performance criteria including determinations of accuracy, precision, linearity, specification, limit of detection, and limit of quantitation. The determination of system suitability requires a series of deliberate variations of parameters to ensure the complete system, that is all instrument(s) as well as the analytical method, is maintained throughout the entire analytical process. Traditionally, method validation has been referred to as “ruggedness” and system suitability as “robustness.”

CIJ: What are the most important aspects of method validation that must be taken into account? 

Susan: In keeping with the FDA guidelines and other accepted criteria, I tend to recommend the International Conference on Harmonization (ICH), particularly Q 2A, which is a widely recognized program that discusses the pertinent characteristics of method validation. This include: method specification, linearity, range, accuracy, and precision (e.g., repeatability, intermediate precision, reproducibility). As mentioned earlier, system suitability is also a critical element and although related to method validation, does require its own protocol.

CIJ: What three areas do you see the laboratory having the hardest time with in preparing for accreditation? 

Susan: My responses to this question assume the laboratory employs appropriate instruments to perform the necessary analyses, and that the laboratory employs personnel with experience and knowledge appropriate to develop test methods and interpret test results.

  • By and large, method validation that is not appropriate to the scope of their intended work. Driving this is an overall lack of information about method validation. Oftentimes there is an assumption that multiple recoveries of CRMs constitute “validation”. While it may be one element, this only demonstrates the instrument’s suitability. My recommendation is to utilize any one of a number of good single laboratory validation protocols. Options include, but are not limited to AOAC International, American Chemical Society, ASTM, and ICH protocols.
  • Second is the lack of statistically sound sampling protocols for those laboratories that are mandated by their governing states to go to the field to sample the product from required batches. Sampling protocols needs to address the heterogeneity of the plant, defining the batch, and determining/collecting a sample of sufficient quantity that will be both large enough and representative of the population, and to provide the laboratory an adequate amount from which to sub-sample.
  • Third, sample preparation. This is somewhat intertwined with my previous point. Once an appropriate sample has been collected, preparation must be relevant to the appropriate technology and assay. It is unlikely that a laboratory can perform a single preparation that is amenable to comprehensive testing.
emerald test retail

Emerald Scientific Proficiency Test Approved for Lab Accreditation & Regulatory Compliance

By Aaron G. Biros
No Comments
emerald test retail

Emerald Scientific’s Inter-Laboratory Comparison and Proficiency Test (ILC/PT) was recently approved in Washington as an official cannabis lab PT program, according to a press release. The Emerald Test program measures the accuracy of individual labs as well as comparing their results to other labs for indicators of variability and performance improvement.

Washington requires certified cannabis labs to participate in proficiency testing and Emerald Scientific’s tests is the only approved program in 4 out of 5 of the categories: potency, pesticide, heavy metals and residual solvent analysis. The most recent round of The Emerald Test showed broad improvements in many of the testing categories.

Perry Johnson, a third-party lab accreditation service for ISO/IEC 17025 also decided that The Emerald Test “meets the audit criteria for the proficiency test participation requirement for the accreditation,’ according to the press release. The proficiency test is a key component of quality assurance, which is a major requirement for labs seeking ISO 17025 accreditation. “The Emerald Scientific PT ensures that the cannabis testing labs are performing their function to the best of their ability,” says Reggie Gaudino Ph.D., vice president of Science, Genetics and Intellectual Property at Steep Hill Labs. “Any lab that isn’t participating and exceeding the minimal passing requirements should be viewed as suspect. It’s that important.”

According to the press release, Emerald Scientific’s spring 2017 program has expanded from 5 to 6 tests. The residual solvents and pesticide analysis portions offer more comprehensive testing that previously. “The other tests include 2 microbial panels and a Potency Test, which measures 5 cannabinoids including THC, THCA, CBD, CBDA, and CBN,” says the press release. “New this spring is the Heavy Metals Test, which is offered in 2 parts, one solution for cannabis heavy metals and the other in a hemp matrix.”

More than 60 labs are expected to participate. Results will be released at the National Cannabis Industry Association’s Cannabis Business Summit and Expo on June 13, 2017. For more information please visit www.emeraldtest.com or email sales@emeraldscientific.com.

emerald test retail

Analyzing The Emerald Test Results: Cannabis Labs Making Progress

By Aaron G. Biros
No Comments
emerald test retail

The Emerald Test advisory panel recently convened to review the results from the Fall 2016 round of the semi-annual Inter-Laboratory Comparison and Proficiency Test (ILC/PT), ahead of the third annual Emerald Conference just a few weeks away. After reviewing and analyzing the results, the panel noticed a significant improvement across the board over their Spring 2016 round of proficiency testing.rsz_emerald-scientific_letterhead-1

Emerald Scientific’s ILC/PT program is a tool laboratories use to check how accurate their testing capabilities are compared to other labs. A lab receiving The Emerald Test badge indicates their testing meets the criteria established by the panel to demonstrate competency. This means that they were within two standard deviations of the consensus mean for all analytes tested, according to Wes Burk, vice president of Emerald Scientific. He says the labs performed better than expected on both the microbial and pesticide tests.

Wes Burk, vice president of Emerald Scientific.
Wes Burk, vice president of Emerald Scientific.

emerald test retailEach lab has access to raw, anonymized data including a consensus mean, z-scores and kernel density plots. This round measured how well 35 cannabis labs perform in testing for potency, pesticides, residual solvents and microbial contaminants such as E. coli, Salmonella, Coliform, yeast and mold.

The advisory panel includes: Robert Martin, Ph.D., founder of CW Analytical, Cynthia Ludwig, director of technical services at AOCS, Rodger Voelker, Ph.D., lab director, OG Analytical, Tammie Mussitsch, QA manager at RJ Lee Group, Shawn Kassner, senior scientist at Neptune & Company, Inc., Jim Roe, scientific director at Steep Hill Labs, Chris Hudalla, Ph.D., founder and chief scientific officer at ProVerde Labs, Sytze Elzinga, The Werc Shop and Amanda Rigdon, Chief Technical Officer at Emerald Scientific.

amandarigdon
Amanda Rigdon, chief technical officer at Emerald Scientific

According to Amanda Rigdon, chief technical officer at Emerald Scientific, the labs performed very well in potency, residual solvents and microbial testing PTs. This is the first year the proficiency testing includes pesticides. “All of the labs did a great job identifying every pesticide in our hemp-based PT, but some more work will most likely have to be done to bring quantitative results in line,” says Rigdon. “Since this was the first pesticide PT we had offered, we were pretty conservative when choosing analytes and their levels. For the most part, analytes and levels were taken from the Oregon pesticide list, which is widely recognized to be the most reasonable and applicable pesticide list out there to date.” They covered pesticides of high concern, like abamectin and Myclobutanil, but also included a wide range of other pesticides that labs are expected to encounter.

Shawn Kassner, senior scientist at Neptune
Shawn Kassner, senior scientist at Neptune & Company, Inc.

Shawn Kassner, senior scientist at Neptune & Company, Inc., believes microbial contamination proficiency testing should be a priority for improving public health and safety going forward. Although five participating labs did not receive badges for the microbial contamination PTs, panel members say the overall performance was really quite good. “Microbiology testing are essential analyses for all cannabis products and it’s just slower in regulatory implementation than potency testing,” says Kassner. “The risk of Salmonella and E. coli to an individual using a medical cannabis product could be very life threatening. Microbiology contamination is a huge concern for any public health agency, which is why we have seen that microbiology testing is usually the first analytical test required after potency.” Kassner notes that there were few outliers and with each Emerald PT program, he is seeing an improvement in overall laboratory performance.

For The Emerald Test’s next round, the panel hopes to make some improvements in the test’s robustness and consistency, like obtaining assigned values for all samples and comparing to a consensus mean. “We want to develop permanent badge criteria, streamline the appeals process and possibly implement a qualitative performance review in the pesticide PT,” says Burk. For the next round of pesticide PTs, they want to build a better list of pesticides to cover more states, allowing labs to pick a set based on their state’s regulations. Burk says they also want to collect data on whether or not matrix-matched curves were used for pesticides.

Rodger Voelker, Cynthia Ludwig and Shawn Kassner, all members of the advisory panel, will be speaking at the Emerald Conference, discussing some of their findings from this round of proficiency testing. The Emerald Conference will take place February 2nd and 3rd in San Diego, CA.