Rev. Proc. 64-4

SECTION 1. PURPOSE.

The purpose of this Revenue Procedure is to prescribe guidelines relative to acceptable sampling procedure in connection with the rules under section 1.453-2(d) of the Income Tax Regulations covering sales of personal property made under a revolving credit plan.

SEC. 2. AUTHORITY.

In Treasury Decision 6682, C.B. 1963-2, 197, prescribing regulations under section 453 of the Internal Revenue Code of 1954, sales made under a revolving credit plan will be treated as sales on the installment plan to the extent provided in section 1453-2(d) of the regulations.

SEC. 3. GENERAL.

.01 The percentage of charges under a revolving credit plan which may be treated as sales on the installment plan in order to compute the gross profit to be deferred is to be computed by making an actual segregation and allocation of charges in a probability sample of revolving credit accounts.

.02 The taxpayer may select one of the following methods for obtaining a sample of accounts:

1 The taxpayer may use any sampling procedures that are in accord with generally accepted probability sampling techniques. The procedures used, however, must be documented and made available at the request of the District Director of Internal Revenue for the district in which the taxpayer's returns are filed. In these procedures the taxpayer must specify the steps taken to satisfy the requirements listed under section 4 of this Revenue Procedure titled `Standards of Probability Sampling.'

2 The taxpayer may elect to use the procedures specified in section 5 of this Revenue Procedure titled `An Acceptable Procedure for Sampling.'

3 The taxpayer may elect to use a 100 percent sample.

SEC. 4. STANDARDS OF PROBABILITY SAMPLING.

.01 If the taxpayer elects to use the procedures specified in section 3.021, the sampling plan which is used must conform to the standards of:

1 `Report of Committee on Standards of Probability Sampling for Legal Evidence-Admissibility of Data from Probability Samples,' which was published by the Society of Business Advisory Professions, Inc., in cooperation with New York University. For convenience, this report is reproduced as an appendix.

.02 In addition to satisfying the requirements of the above referenced report, the sampling plan must meet the following requirements:

1 The sampling procedure and sample size must be such as to provide for a relative sampling variability in the amount of gross profit to be deferred ( i.e. , the coefficient of variation of the estimate expressed as a percentage) of no more than two percent.

2 The procedures used and the records preserved must be such as to permit verification at a later date of the sample selection and the computations performed with sample accounts.

3 The procedure used and the records preserved must be such as to permit verification that the frame from which sampling was done adequately covers the population at the time of sampling.

.03 The population from which the sample is to be selected is all accounts under a revolving credit plan as defined in section 1.453-2(d)(1) of the regulations. In these guidelines, such accounts are referred to as revolving credit accounts. In creating a frame, the population may be partitioned, clustered or otherwise organized in order to increase sampling efficiency.

.04 Each sample account shall be classified as either-

1 A null account:

(a) accounts with zero balance;

(b) accounts that are `disregarded' under section 1.453-2(d)(2)(i) of the regulations;

(c) an account that does not qualify as a revolving credit account, e.g. , a traditional installment account or a 30-day charge account; or

2 An allocable account: All other accounts, e.g. , a revolving credit account with a nonzero balance, except a `disregarded' account.

.05 For allocable accounts, the segregation of the charges into those treated as sales on the installment plan and those not treated as sales on the installment plan must be in accordance with the rules of section 1.453-2(d) of the regulations. The nulls and the results of the segregation should be recorded on a worksheet in the format of Exhibit 1, page 653. Data to be submitted with the tax return shall consist of the following:

1 A statement of the sample design. This statement must include:

(a) Description of the frame or frames.

(b) Definition of the sampling unit or units.

(c) The procedure for drawing the sampling units.

(d) Computations on which the decision of sample size was based.

(e) The formula or procedure for calculating the relative sampling variability in the estimate of gross profit to be deferred.

2 Computation of the amount of gross profit to be deferred. This computation is to be submitted on a worksheet in the format of Exhibits 2 and 3, pages 654 and 655, respectively, except that the sample design need not be a replicate type. However, if the taxpayer is not able to demonstrate to the satisfaction of the District Director that income from sales on the installment plan is clearly reflected by considering all sales as being made in the taxable year, appropriate adjustments must be made in the computations provided in the worksheet of Exhibits 2 and 3 in order to conform to section 1.453-2(d)(6)(vi) of the regulations.

3 Computation of the relative sampling variability (derived from the sample itself) in the estimate of gross profit to be deferred.

SEC. 5. AN ACCEPTABLE PROCEDURE FOR SAMPLING.

.01 Definition of Population to be Sampled .-The population from which the sample is to be selected is all accounts under a revolving credit plan as defined in section 1.453-2(d)(1) of the regulations as of the end of the last billing month ending within the taxable year. In these guidelines, such accounts are referred to as revolving credit accounts.

.02 The Frame from which Sample is Selected .-A listing of all accounts that are designated as revolving credit accounts that cover at least the population defined above must be prepared for selection of the sample.

1 It is preferable to list only revolving credit accounts. The frame may include accounts that are not revolving credit accounts if the normal listing source also contains various types of other accounts, e.g. , 30-day charge accounts and traditional installment accounts, intermixed with revolving credit accounts.

2 It is preferable to keep zero balance accounts to a minimum. If inactive accounts are removed at intervals, a statement must be included with Exhibit 3 specifying when inactive accounts are removed and the definition of an inactive account. If any other procedure is followed for the removal of zero balance accounts, that procedure must be specified.

3 The list may exclude certain types of revolving credit accounts, if these accounts are incorporated 100 percent as provided by section 5.06, which describes the method of including these special accounts in the final determination of the amount of gross profit to be deferred.

EXAMPLE: Assume the normal credit limit is $300, but larger amounts are permitted for particular customers. The taxpayer may elect to exclude all of these accounts from the list and incorporate them 100 percent in the final determination.

4 For many taxpayers the most convenient source for the list will be the addressing plates used in billing and mailing. A continuous listing on one roll of paper is preferred. If separate sheets of paper are used, care must be exercised to identify the proper order of the sheets and to maintain this order when sampling. This listing is to be preserved as part of the records.

5 The taxpayer may elect one of the following methods of sequencing the list or lists of the frame:

(a) Ascending or descending sequence by account number for all establishments of the taxpayer which maintain the accounting records for revolving credit accounts (hereafter referred to as establishments).

(b) Alphabetic sequence of customer name for all of the establishments of the taxpayer

(c) Ascending or descending sequence by account number within each of the establishments of the taxpayer.

(d) Alphabetic sequence of customer name within each of the establishments of the taxpayer.

(e) A combination wherein some establishments have accounts sequenced in accordance with (c) and other establishments have accounts sequenced in accordance with (d).

(f) Where the taxpayer can demonstrate that prior valid accounting practice justifies it, he may follow a variant procedure but he must include a statement with Exhibit 3 describing his method of sequencing the list or lists.

6 Each list must be serially numbered after sequencing. A taxpayer who sequences in accordance with method (c), (d), or (e) may elect to begin the serial numbering anew at each establishment ( i.e. , a separate list is prepared for each establishment) or he may elect to continue the sequence through all establishments.

7 A taxpayer shall not change the elections he has made, as specified under paragraphs 1 through 6, above, without obtaining the consent of the District Director.

.03 Definition of the Sampling Unit .-A sampling unit consists of an account as described under section 5.02, above; i.e. , each account listed in the frame is a potential sampling unit.

.04 Selection Procedure.-Using the list or lists which comprise the frame, a sample is to be selected in the following manner:

1 Obtain the total number of accounts in the frame.

2 The minimum sample is as follows:

         Total Accounts                           Accounts required

3 The initial sample consists of the minimum sample plus additional accounts which are required if there are nulls in the frame. To determine the number of additional accounts required, estimate the portion of the frame consisting of allocable accounts and increase the minimum number of sample accounts by dividing by the square of the estimated portion. In this example, if we estimated that 1/2 of the accounts are allocable, we square 1/2 ( i.e. , we multiply 1/2 by 1/2 ), to obtain 1/4 and divide 754 by 1/4 to obtain 3,016 as the initial sample. In this sample, 2,262 additional accounts are required (3,016 minus 754). In another example, if we estimate that 9/10 ths of the accounts are allocable, we square 9/10 ( i.e. , we multiply 9/10 by 9/10 to obtain 8 1/100 and divide 754 by 81/100 to obtain 931 as the initial sample. In this example, 177 additional accounts are required (931 minus 754). This latter example is carried forward in the following illustration.

4 A `skip interval' is used to obtain the sample from the frame. To obtain the skip interval, multiply the total number of accounts in the frame (step 1) by five and divide the product by the number of accounts in the initial sample (step 3). The taxpayer may reduce the skip interval if a smaller skip interval is more convenient. If taxpayers with few revolving credit accounts find that the skip interval is less than 6, then all accounts will be included in the sample. See section 3.023, above. Continuing the example for a taxpayer with 125,643 accounts in the frame and an initial sample of 931 accounts, the skip interval is 674 since 125,643 x 5 / 931 = 674. For convenience, this may be reduced to 650 accounts.

5 Five random starting points will be needed in connection with the skip interval in order to draw the sample accounts. These random starting points may not be greater than the skip interval and may not be duplicates. To obtain the five random starting points, refer to Table 1, `List of Starting Points,' page 656, and find the section for the skip interval determined in step 4. Each of the ten sections of this table has ten columns. Determine the column to use as follows:

Add the last digit of the calendar year to the last digit of the Employer Identification Number or Social Security Number as shown on the income tax return. If the sum is 9 or less, use that column. If the sum is greater than 9, subtract 10 from the sum and use the column indicated by the remainder. For example, if the Employer Identification Number is 345679 and the year is 1963, the sum is 3 + 9 = 12. Since this sum is greater than 9, subtract 10 to obtain 2, and use the column 2 in the seventh section of Table 1.

6 If any starting point is greater than the skip interval or is a duplicate of a previous number, omit it and go on to the next one. If necessary, use the next column. If this is column 9, continue with column 0 for the same section of skip intervals. In the example, the first two random starting points listed in column 2 for skip intervals 500-999 are 577 and 169. Omit the next since 672 is greater than 650 and take the next three, viz. , 193, 021, and 116.

7 Prepare a worksheet in the form of Exhibit 1. Enter the digit 1 for subsample number, the skip interval and the first starting random number. Also enter the same number on the first line in the first column. Add the skip interval to the starting random number to obtain the next account in this subsample and enter this number on the second line. Adding the skip interval once again will supply the entry for the third line and the process is continued until the end of the frame is reached. In the example, the first starting random number is 577, which is also entered on the first line under `List serial number,' column (1). The entry on the second line of column (1) is 1227 (577 + 650 = 1227); on the third line, 1877; the fourth line, 2527; and so on. The last entry will be 125,377 (since the frame has 125,643 accounts).

The second Exhibit 1 is prepared in a similar way, using the second starting random number and the same skip interval. Continuing the example, 169 is entered as the starting random number and is also posted on the first line under column (1). The entry for the second line of that column is 819; for the third line, 1469; for the fourth line, 2119; and so on. The last entry will be 125,619.

8 If a separate list is prepared for each establishment (see section 5.026 above), the same random starting numbers and skip interval are to be used for each list (corresponding to each establishment with a separate list) from which sampling is done in any one year.

9 For each account entered on an Exhibit 1, locate all of the pertinent account history data and obtain the figures for columns 4, 5, 6 and 7 of the worksheet.

(a) The first column has already been entered in accordance with paragraph 7, above.

(b) Column (2) requires the name of the account holder. Enough of the name should be shown to permit positive identification of the account.

(c) Column (3) requires the address of the account holder.

(d) Column (4) requires the revolving credit balance as of the last billing date of the fiscal year. If the account is not a revolving credit account, do not enter any amount but designate the type of account, e.g. , `traditional installment' or `30-day.'

(e) Column (5) is to be used for balances that are `disregarded' in accordance with section 1.453-2(d)(2)(i) of the regulations.

(f) Column (6) consists of charges that are treated as installment sales in accordance with the rules of section 1.453-2(d) of the regulations.

(g) Column (7) consists of charges that are not treated as installment sales.

.05 Estimating Procedure .

1 After Exhibits 1 have been prepared for each of the five subsamples for the initial sample, Exhibit 2 is completed for the business as a whole.

(a) Post from Exhibit 1 for each subsample the total of column 6 to line 1 of Exhibit 2 and the total of column 7 to line 2 of Exhibit 2. Add lines 1 and 2 to obtain line 3 and divide line 1 by line 3 to obtain a percentage which is posted to line 4.

(b) Both the highest and lowest percentage of the five subsamples are posted to line 5. The difference between these two amounts is divided by five and entered on line 6. The entry for line 7 is obtained from Table 2, `Acceptability Test Level by Number of Accounts,' page 657. For line 8, divide the amount posted to line 6 by the amount posted on line 7. Referring to Table 3, `Number of Additional Subsamples Required,' page 657, determine the number of additional subsamples required and enter this amount on line 9.

(c) For example, the highest percentage of charges treated as installment sales turned out to be 75.2, which was observed for subsample 4. The lowest percentage, 64.6, was observed for subsample 2. The difference is 10.6 percent, which is divided by 5 to obtain 2.12. This amount is posted to line 6. For 125,643 accounts, Table 2 provides an acceptability test level of 2.0, which is posted to line 7. The entry on line 6, which is 2.12, is divided by 2.0 and the quotient, 1.06 posted to line 8. Referring to Table 3, we find that since 1.06 is `over 1.00 up to 1.10,' one additional subsample is required.

2 If an additional subsample is required by paragraph 1 above, prepare an additional Exhibit 1 for each such additional subsample in a manner similar to that specified in section 5.047, above. In order to obtain additional random starting points, refer to section 5.045 and use the next random number, providing it is neither in excess of the skip interval nor a duplicate of a preceding number.

(a) For example, if one additional subsample is required, the number following 116 is 678, which exceeds the skip interval. The number following is 355 and this number is entered on an Exhibit 1 as the starting random number for the sixth subsample. In this case, 355 is entered as the first line in column (1); 1005, as the second line; 1655, as the third line, and so on. The last line is 125,155.

(b) The information from each additional Exhibit 1 should be carried forward to Exhibit 2 by adding one column for each additional subsample.

3 Continue Exhibit 2 by entering additional information from Exhibits 1 for each of the subsamples:

(a) The aggregate of account balances for the last billing date of the fiscal year (from Col. (4) of Exhibit 1) is entered on line 10 of Exhibit 2.

(b) Review each entry in column 4 of Exhibit 1 to determine the largest single balance and enter this amount on line 11 of Exhibit 2.

(c) Determine the number of accounts in each Exhibit 1 and enter the number on line 12 of Exhibit 2.

(d) Enter on line 13 of Exhibit 2, the number of null accounts, determined from Exhibit 1, columns 4 and 5.

(e) Exhibit 2 can be completed at this stage by taxpayers who have not elected to exclude any revolving credit accounts as provided by section 5.023, above. Taxpayers who have elected to follow the procedure of section 5.023 should refer to section 5.06 for the method of including these accounts.

4 Complete Exhibit 2 by adding across lines 1, 2, and 3 for all columns and entering the total in the column headed `Total of all subsamples.' Line 4 is obtained by dividing the total for line 1 by the total for line 3. This entry will be carried forward to Exhibit 3.

Similarly, lines 12 and 13 are added across for all columns and entered in the `total' column. This entry should be referred to when the taxpayer estimates the proportion of nulls in applying the method in a later year. See section 5.043, above.

5 Prepare Exhibit 3 by entering on line 1 the aggregate of charges under the revolving credit plan appearing in the year-end balance reduced by the amount of nonpersonal property sales to which the sample percentage is to be applied. See section 1.453-2(d)(5) of the regulations. Post to line 2 the percent of revolving credit charges to be treated as installment sales. This is the percentage shown in the total column, on line 4 of Exhibit 2. Multiply the amount on line 1 by the percentage on line 2 and enter the product on line 3 as the `Portion to be treated as installment sales.' On line 4 enter the gross profit percentage. Multiply the entry on line 3 by the gross profit percentage (from line 4) and enter the product on line 5. This is the amount to be deferred.

EXAMPLE: If $1,900,000 is the aggregate of charges under the revolving credit plan appearing in the year-end balance to which the sample percentage is to be applied, this amount would be entered on line 1. Multiplying by 70 percent (from line 2), we obtain $1,330,000 as the portion to be treated as installment sales (line 3). If the gross profit percentage is 40 percent, the gross profit which is to be deferred is $1,330,000 x 40 percent = $532,000.

.06 Inclusion of Certain Accounts .-Accounts that were excluded from the frame, as provided by section 5.023, must be incorporated following the procedure set forth in this section.

1 Prepare an Exhibit 1 listing each account excluded from the frame and completing the information required by columns one through seven. Obtain the totals of columns (6) and (7). For example, 192 excluded accounts are listed that show for column (6), $325,000; for column (7), $16,250.

2 Multiply the total for column (6), obtained as a result of step 1, by the number of subsamples used in evaluating the frame (at least 5, see section 5.051, above). Divide the product by the skip interval used (see section 5.044, above). Enter this amount at the end of column (6). This amount will also be carried forward to Exhibit 2.

EXAMPLE: If the total for column (6) of Exhibit 1 was $325,000, multiply this by 6 (since there were six subsamples), and divide by 650 (the skip interval) to obtain $3,000.

3 Repeat the procedure for the total of column (7) of Exhibit 1, entering the quotient at the end of column (7). This amount will also be carried forward to Exhibit 2. For example, if the total for column (7) of Exhibit 1 was $16,250, then $16,250 x 6 / 650 = $150.

4 Add a column to Exhibit 2 and list on line 1 the quotient derived from step 2, above. Enter on line 2 the quotient derived from step 3, above. Add the quotient on line 1 to the quotient on line 2 and enter the sum on line 3. Do not make any other entries in this column.

5 Add the columns of line 1, including the five minimum subsamples, any additional subsamples required by section 5.051(b), and the quotient obtained from section 5.062. Enter the total in the column headed `Total of all subsamples' in line 1. For example, if the five initial subsamples yielded $49,000, the sixth subsample $11,000, and the amount derived from the 192 accounts that were excluded from the frame was $3,000, then the amount to be entered under `Total of all subsamples' is $63,000 ( i.e. , $49,000 + $11,000 + $3,000 = $63,000).

6 Add the columns of line 2, including the five minimum subsamples, any additional subsamples required by section 5.051 and the quotient obtained from section 5.063. Enter the total in the column headed `Total of all subsamples' in line 2. For example, if the five initial subsamples yielded $22,850, the sixth subsample $4,000, and the quotient derived from the 192 accounts that were excluded from the frame was $150, then the amount to be entered under `Total of all subsamples' is $27,000 ( i.e. , $22,850 + $4,000 + $150 = $27,000).

7 For the `Total of all subsamples' column, add line 1 to line 2 and post the sum in line 3. Divide the entry in line 1 by the entry in line 3 and post the quotient as a percentage on line 4. Also post the quotient on line 2 of Exhibit 3. Continuing the example of the two preceding paragraphs, $63,000 plus $27,000 is $90,000, which is posted on line 3. Dividing $63,000 by $90,000, the quotient is 70 percent, which is entered on line 4. The entry of 70 percent is also carried forward to line 2 of Exhibit 3.

8 Similarly, lines 12 and 13 are added across for all columns and entered in the total column. This entry should be referred to when the proportion of nulls is estimated in applying this method in a later year (see section 5,043, above).

9 For Exhibit 3, see section 5.055, above.

.07 In using the worksheets of Exhibits 2 and 3, if income from sales on the installment plan is not clearly reflected by considering all sales as being made in the taxable year, appropriate adjustments must be made in the computations provided in the worksheets of Exhibits 2 and 3 in order to conform to section 1.453-2(d)(6)(vi) of the regulations.

.08 Data to be Submitted with the Tax Return .-The taxpayer must submit with his tax return worksheets in the format of Exhibits 2 and 3.

.09 Records to be Preserved .-The taxpayer must preserve the following records:

1 All lists which constitute the frame from which sampling was performed.

2 Detailed records of individual customer accounts.

3 Worksheets in the format of Exhibit 1.

SEC. 6. EFFECT ON OTHER DOCUMENTS.

In view of the factual nature of the problems involved, the Service will not make an advance determination as to the acceptability of sepcific sampling methods. Revenue Procedure 62-32, C.B. 1962-2, 527, is hereby supplemented by adding to the areas in which rulings will not be issued, as listed in section 3.01, the following paragraph:

27. Section 453.-Revolving Credit Sales as Installment Sales.-Whether a proposed sampling procedure will be acceptable by the Internal Revenue Service for the purpose of determining the portion of revolving credit balances appropriately to be treated as installment account balances.

NOTE: The Exhibits appearing herein will not be separately printed or stocked by the Internal Revenue Service. They are merely furnished as formats.

                              EXHIBIT 1

² Total of column 7 of worksheet of Exhibit 1.

³ Total of column 4 of worksheet of Exhibit 1.

          EXHIBIT 3. __ Amount of gross profit to be deferred

STANDARDS OF PROBABILITY SAMPLING FOR LEGAL EVIDENCE ¹

ADMISSIBILITY OF DATA FROM PROBABILITY SAMPLES

Scope and purpose .-The standards that follow are intended to govern in legal evidence the presentation and interpretation of data obtained by probability sampling as defined later. Other forms of sampling or partial surveys or enumerations are not included in this document. The use of data produced by one of these other methods should be justified by criteria appropriate to its nature and to the circumstances.

These standards establish specifications for sampling, which if certified as followed, and provided the standard error of the sampleresult is small enough to be innocuous, should establish the acceptance of the numerical result of a sample as legal evidence, on the same status as the result of an equal complete coverage of all the units in the same frame (q.v.) whence the sample came, without the requirement of a dissertation on the theory, techniques, meaning and reliability of sampling.

An equal complete coverage is a coverage of all the sampling units (a 100%sample) in the same frame as was used for the sample, under the stipulation that the complete coverage was carried out with the same procedure as was used for the sample for eliciting the information, with the same thoroughness, and over about the same period of time.

Disclaimers .-These standards do not specify what kinds of information might be useful or admissible in any case; nor do they deal with the methods, definitions and techniques by which such information is to be elicited from people, business establishments, or records; nor do they describe methods for testing industrial product; nor do they lay down the criteria for making an audit of a financial statement by professional accountants. They offer no criteria by which to evaluate the usefulness or the scientific value of any data. They offer no criteria by which one decides whether a given frame will give adequate coverage of the universe that one desires to study. Judgment on these matters must come from experts in the subject-matter and from experts in the techniques of interviewing and testing. Statistical theory can not accomplish this judgment, although experiments designed with the aid of statistical theory will estimate the numerical differences between the results to be expected from different methods, definitions, and techniques, or between two sets of inspectors or interviewers. Such questions are the same whether the survey is to be conducted by a sample, large or small, or by a complete coverage.

Basis of admissibility .-If a study would be admissible evidence as a complete coverage of a certain frame, then it will also be admissible if it is carried out as a probability sample of that frame.

a. An exception occurs if the precision of the sample is too low for the purpose, or if flaws in execution render the results questionable.

b. In most respects, the preparation, execution, interpretation, and use of survey-data are the same whether the survey be a complete coverage or a probability sample. The problems created by a gap between the frame and the universe, by the questionnaire, by the procedure of testing, or interviewing, or by the coding, the interpretation, and the judgment of the expert on the usefulness or the propriety of the data for the problem at hand, are the same for a sample and for an equal complete coverage.

c. The chief difference is that data from a sample possess error from the use of sampling in place of a complete coverage of the same frame, but the margin of error with its associated probability is calculable and is known objectively; it is not a matter of judgment nor of expert opinion, but is a mathematical consequence.

d. Another difference is that the quality of the testing, interviewing, supervision, coding in a sample-survey can be superior to the quality of a complete coverage, and the accuracy of the sample-survey can thus be the greater, provided one takes advantage of the smaller size of the job.

Form of the results .-The result of a sample will be in the form of a numerical upper (lower) limit that one may accept, with a stated probability, as above (below) the result that would have come from an equal complete coverage of the same frame. The result of a sample is sometimes in the form of both an upper limit and a lower limit that one may accept, with a stated probability, as including the result of the equal complete coverage.

Definition .-Probability sampling is a procedure by which one obtains a result from a selected portion of a total number of establishments, accounts, or other items, that will be the same, within calculable limits of uncertainty, as the result that would have come from an equal complete coverage, viz., an examination of all the establishments, accounts, or other items in th frame with the same definitions and care as were expended on the sample. The special feature of probability sampling is that it permits use of the theory of probability for the computation, from the sample itself, of the margin of error and the probability associated therewith that has been introduced by the use of a sample in place of an equal complete coverage.

The frame .-The objective inferences in regard to the precision of a sample refer to the frame, but owing to gaps between the frame and the universe, wherein the frame fails to cover the universe, these inferences do not necessarily apply to the universe. To be acceptable, the frame whence a sample will be drawn, together with the accompanying procedures for covering any sampling unit once drawn from the frame, must yield, when the sample is increased to 100 %of the sampling units of the frame, a satisfactory coverage of the universe that one desires to study.

a. The frame is composed of sampling units, the totality of which, if covered by the survey, would constitute a 100%sample. The frame is often in the form of a list of sampling units. Each sampling unit in the frame may contain people, firms, product, or records, that are capable of furnishing the information that is desired in the study.

b. A frame is a means of access to the universe. Without a frame there can be neither a complete coverage nor a probability sample.

c. Sampling can not overcome the fundamental defects of a frame, nor of any procedure for using it. Any hazard that arises from incompleteness in the frame or from any proposed method of covering a sampling unit will persist undiminished, no matter how big the sample, even if it includes all the sampling units of the frame. If this hazard is great enough, the frame may not be suitable for either a complete coverage or a sample.

THE PRECISION OF A RESULT

The index of precision .-The index of the precision of any estimate is its standard error, or an objective estimate thereof. The margin of difference between an estimate made from a sample, and the result that would have been obtained from an equal complete coverage, is usually expressed as an appropriate multiple of the estimate of the standard error, for a specified probability or risk.

The limits of sampling error .-The maximum uncertainty in any estimate that is made from a sample used in place of an equal complete coverage of the same frame may be placed at 3 times the standard error of the estimate.

a. The actual difference between (a) the estimate produced by the sample and (b) the result of a complete coverage carried out under the same conditions, is usually much less, and may be in either direction. Only about 1 estimate in 40 will fall below 2 standard errors from the result of a complete coverage, and only about 1 in 40 will fall above 2 standard errors.

b. If the sample is very small (below 25 independent sampling units), or if the distribution in the frame is highly skewed, the calculations of the margin of error, also its interpretation, should be carried out with the aid of the theory of probability, with regard for the distributions involved.

c. This interpretation of a statistical calculation such as the standard error is not a matter of opinion, nor of judgment, but is a mathematical consequence.

d. Every characteristic estimated from a sample has its own standard error, and any standard error depends on the sample-design that was used.

Size of sample .-The size of a sample is not sufficient information by which to calculate the standard error of a result. The procedure for the selection of the sampling units, and for the calculation of the estimates, are equally as important as the size.

The size of a sample is not by itself a criterion of its precision, nor of its accuracy, nor of its usefulness.

Information provided by the standard error .-Once a sampling survey is completed, and an estimate is obtained therefrom, the standard error of this estimate, or the margin of error and its associated probability, calculated (a) by the proper statistical theory from the results of the sample, and (b) under the necessary conditions for validity (use of random numbers, adherence to the prescribed sampling procedure) conveys all the information that there is in regard to the precision of the estimate in question.

Information not provided by the standard error .-A standard error, or a margin of error, does not however convey all the information that there is in regard to blemishes in the execution of the sampling procedure.

Details concerning the sampling procedure, such as the modes of stratification, and the statistician's reasons for his choices of plan and for the size and description of the sampling units, convey no new information about the precision of an estimate, once the numerical value of its standard error is known. The only reason for including such information in the testimony is to permit probe of the execution of the sampling procedure.

RANDOMNESS

Random numbers .-A system of selection that depends on the use in a standard manner of an acceptable table of random numbers shall be accepted as random.

Other devices .-Mechanical devices that depend on physical mixing or shuffling are hazardous and are not acceptable as random without proof, which will require the aid of statistical techniques. The words `thorough mixing,' for example, have no standard meaning.

Any method of selection that merely removes the choice of the sampling units from the judgment of the interviewer, without giving definite relative probabilities to each unit, does not constitute a random selection, and does not produce a probability sample.

THE STATISTICIAN'S TESTIMONY

The statistician's testimony should include a statement of his responsibility in the engagement: what it included, and what it did not include.

The sample-design and the instructions for carrying it out should be written, and should be available, to be made part of the evidence, if a probe of the execution of the sample-design if required. The written sample-design and instructions should include:

a. The frame or frames. If a frame is supplanted in some of the steps by rules for the allotment of serial numbers to certain units, these rules should be written, in order that the selection and identification of these units may be retraced. (Files or other material that are necessarily in constant use may change from day to day and render retracing difficult or impossible.)

b. The method of defining and of covering a sampling unit.

c. The procedure for drawing the sampling units. The rules for the identification and for the coverage of a sampling unit.

d. The source of the random numbers and the method of using them.

e. The formula or the procedure for forming the estimates.

f. The formula or the procedure for calculating the standard errors of these estimates (by which to compute the margin of error for any estimate).

The interviewers' or the inspectors' original forms, and the cards or work-sheets derived therefrom, should be filed so that it is possible to retrace the transfer of information from the original entries through the tabulations, and to repeat any calculations for the estimates or for their standard errors.

There should be evidence to show whether the serialization of the sampling units in the frame was carried out independently of the drawing of the random numbers. This condition is satisfied if the frame and the serialization of the sampling units therein were completed prior to the drawing of the random numbers. If the random numbers were drawn by the survey organization, the starting point in the table and the method for abstracting the random numbers therefrom should be specified by the consulting statistician so that the selections may be retraced.

The statistician's testimony should include a statement of steps that he took to ensure understanding of the procedure, and tests of compliance.

It should describe:

a. The steps that he took to measure-

i. the proportion of failures to cover the sampling units properly and in the manner prescribed;

ii. the proportion of sampling units not intended for the sample but covered or partly covered by mistake;

iii. the errors and difficulties in reporting;

iv. nonresponse;

v. slips and blemishes in the execution of the sampling procedure;

vi. surprises in the material sampled, and their possible effects on the results and on their standard errors (e.g., the discovery of large accounts in the sample, when none was suspected when the sampling plan was drawn up).

b. Their possible effects on conclusions drawn or to be drawn from the data.

SOME POINTS ABOUT SAMPLES

(For information only; not a part of the standard)

The standard error does not measure the accuracy nor the usefulness of a result. These, as stated earlier, depend on the information elicited, and are the responsibility of counsel, acting with the aid of experts in the subject-matter. The standard error, however helpful in the use of data from samples, measures only the margin of sampling error-the difference between the result of the sample and the result that would have been obtained from an equal complete coverage.

Sampling can not overcome fundamental defects in definitions and in methods for eliciting information. In fact, these defects afflict complete coverages and samples alike. They can be measured by proper experimentation, carried out with the aid of statistical theory. In the absence of experimentation to determine their magnitudes, the judgment of the statistician based on experience in previous surveys may be helpful until later surveys and experiments render definitive measures.

a. A small standard error of an estimate merely means that an equal complete coverage would have given a result very nearly the same as the estimate from the sample.

b. A small standard error does not indicate successful coverage; it does not indicate that the field-work was good, nor that the errors of nonresponse were small. The standard error does not measure the biases of nonresponse, of wrong information, nor of other errors that may afflict complete coverages and samples alike.

c. It is possible for a result to useful and still to possess a wide standard error. A result obtained by definitions and techniques that have been drawn up with care, based on experience, and by excellent interviewing and supervision, may have a wide standard error because the sample was small; yet such a result may well be preferable to one obtained with a bigger sample, with a smaller standard error, but whose definitions, techniques, and interviewing were out of line with the best practice.

A sample-design may be valid though inefficient. Validity and efficiency are unrelated. A standard error thus retains its validity and interpretation even though some other method of sampling, known today or to be discovered tomorrow, turns out to be more efficient than the plan actually used.

The number of sampling units in the frame is of little consequence to the precision of a sample. In fact, the size of the frame is usually ignored in calculations of the proper sample and in calculations of the standard error of a result, except when the frame is almost as small as the sample.

Thus, the size of sample required to reach a prescribed precision for the valuation of a small lot of material will be about the same as for a big lot. The size of sample for a small city will be about the size required for a big city or for the whole country, except for possible greater variance between the sampling units in the bigger frame.

¹ Reprinted from Special Report, Current Business Studies of The Society of Business Advisory Professions, Inc., in cooperation with the Graduate School of Business Administration of New York University. Title `Report of Committee on Standards of Probability Sampling for Legal Evidence-Admissibility of Data from Probability Samples.'