General comments
This is a relevant and informative review of cost-effectiveness studies in ACL, taking into account several important decision problems in this field. However, it is difficult to get an idea of what is the novelty of your review without including previous reviews conducted in this field to the introduction and/or discussion sections.
With regards to length of manuscript, both introduction and methods sections are quite short, but it should potentially be possible to abridge in the combined results/discussion section without losing information, if you need to restructure to increase word count in the shorter sections.
The language is acceptable (however, acceptable is not enough), and the text would benefit from overviewing the structure (specified below) and by thorough check by a native English speaker. Moreover, the manuscript is too long and needs to be shortened.
Abstract
Would it be possible to develop the description of methods and results? There is for example currently no description of the search strategy in the methods or overview of findings in the results section (thus the conclusions do not clearly follow from the results). I believe the introduction provided under the purpose heading could be shortened to allow some more words for these other sections.
It is sometimes a bit unclear if you are focusing on young adults or adults in the review (compare e.g., title, introduction to purpose and aims statement). Please add Level of Evidence
Title
Would it be possible to add the information that this is a review in the title? In other words, the most important finding in one sentence
Introduction
I would suggest to add at least some of the recently conducted literature reviews on this subject to the references in the abstract (such as Ferrari et al. Outpatient versus inpatient anterior cruciate ligament reconstruction: A systematic review with meta-analysis. Knee. 2017 Mar;24(2):197-206. and Saltzman et al. Economic Analyses in Anterior Cruciate Ligament Reconstruction: A Qualitative and Systematic Review. Am J Sports Med. 2016 May;44(5):1329-35.). In particular since you write in row 42 that there is “currently” a lack of robust trials (but refer only to references from 2011 or older, as far as I can see), while still saying in row 54 that the evidence has “grown”.
Would it be possible to provide figures in a more comparable manner (e.g., 200,000 procedures performed annually in the US vs 13.5 reconstructions per 100,000 in the UK)?
Row 38: What is meant by “as a marker”? Do they only measure indirect costs?
Row 44-47: You make a statement here about professional athletes. Do you have a reference saying that these procedures are not paid for by NHS, and where the limit goes to what is a professional athlete in such studies? Do you have any indication that the costs estimated in such studies are not relevant for your tax-based system, and would that not transfer also to other studies conducted in privately/insurance paid health systems (to be added as a an exclusion criteria)?
Row 54-57: Are you saying that there is no evidence on the clinical outcomes, but increasing evidence on the cost-effectiveness? Can you please clarify this a bit? Sounds strange to have information on health outcomes accessible to assess cost-effectiveness but not being able to say anything about the health outcomes in isolation. Or do you mean that it has become only a question of rationing and withdrawing services to save money (cost without effectiveness)?
Row 62-77: This part is not really described in a traditional way, and may be confusing for many readers. Could you clarify better in the text the differences between cost-effectiveness, cost-utility and cost-benefit analyses? Also, potentially add some more methods oriented references from health economics, where readers can learn more?
Row 74-75: I have not seen the distinction you make between cost beneficial and cost-effective before (cost beneficial have in those papers I have read referred to results from cost-benefit analysis), but I suppose what you here call cost beneficial is what is often called “the dominant” treatment? However, could you please clarify your description of what is cost-effective? Now you at first say that it is not the same as being both better in terms of costs and health outcomes (which would be called cost beneficial according to your definition), but thereafter it appears as if you say that any treatment (regardless of the relationship between costs and health outcomes) would be termed cost-effective, as long as it is possible to rank/compare it to other treatments (sounds more like a question of which health measure is used). Maybe you can clarify this in relation to e.g., threshold value?
Material and Methods
Please start this section with the IRB-approval.
Row 90: It looks as if this row of text is not complete (no full stop)?
Row 93: Can you add explanations (e.g., written out names) to the databases used?
Rows 97-98: Can you please clarify your search structure, maybe adding a table, so that it can be repeated? Now it is unclear what terms were actually used, since neither Economy nor QALY are MeSH-terms (I would have expected either “Cost-Benefit Analysis”, or “Costs and Cost Analysis” to be the “economics”-term, as both are MeSH-terms)? It is also unclear how these terms were combined or what were the number of hits etc.
Can you clarify how you handled inclusion and exclusion criteria? This appears to have been done in different steps (at first only inclusion to get the 17 articles of interest in PubMed and thereafter applying exclusion criteria)? I would recommend that you clarify also in you figure 1 how the exclusion and inclusion happened, preferably using the Prisma flow diagram as a guide.
Rows 100 and 108: Am I correct that from the PubMed search, the identified paper that was in a different language than English was immediately excluded, while the papers identified through other searches were read by a native speaker to assess their possible relevance?
How was level of evidence judged? I expect that this was in line with the journal instructions, but if so, could you add a reference or briefly explain the principles?
Results and Discussion
First of all, you need to separate Results and Discussion.
Row 117-onwards: I would suggest to change the heading to clarify that this is the Results and discussion.
A general comment to the combined Results and Discussion section is that it is difficult to see what the results are and what is interpretation/discussion. I would suggest that you revise this part of the manuscript so that you e.g., start by introducing the decision problem being discussed (you already do this in most of the sections), thereafter present results found for that decision problem, and end with a discussion of those results (with own interpretation and commenting on why results differ between the included studies and e.g., previous review results and similar findings in papers not included as results due to not fulfilling inclusion/exclusion criteria). This would also make it clearer for readers how they should judge the evidence available.
Sometimes you report, from e.g., a reference, the results for each compared treatment, and sometimes you report the difference between compared treatments (this also applies to the tables where you present results). Would it be possible to make it easier for the reader in some way?
I found it a bit difficult to follow the terminology in some aspects, e.g., you write about “knee instability” in one section and in another you report instead “failure rates”. Is this the same thing, or what is meant by failure rate?
Rows 176-178: Can you comment/clarify your views of this reported result? I would expect that readers not familiar with econometric modelling will have difficulties interpreting this.
Rows 257-258: What should be viewed as a comparable result to the other reported interventions, the cost per operation or the cost per hour (my main take home message from this reported result is that the operation must be very quick)?
Row 279: Do you have any suggestion on what is the “best option” when it comes to such scales, and what would be the correct statistical method to employ?
Rows 282 and 284: Here are reported to figures in QALY. Should these be interpreted as change in QALY between two compared interventions, or what is it? And can you clarify the reported result that those with higher QALY (which should be better) had more pain?
Row 305: How did they demonstrate that this was the most cost-effective strategy?
Row 378: Can you clarify during which time this reported “no difference” was reported? I would expect that it is very important to relate all comparisons between in-patient and out-patient surgery to the time period for which costs and health outcomes were being studied (as it appears to be self-evident that it is more costly to have someone staying in hospital than going home directly after the procedure is finished)?
Row 383: Do you have a reference for this in ACLR? For some other procedures I have heard suggested that this is not the case, but that insurance-based systems may result in the more resource intense procedure is chosen (because the insurance company do not question the use of such a resource and the user expects to receive the best they can get based on their insurance status). Also, tax-based systems (for example the Swedish system) has shown a high pressure to decrease number of nights in hospital, resulting in a health system with a very low number of hospital beds per capita.
Row 385: How should these results (for BPTB vs HA) be interpreted with regards to day surgery vs inpatient care? Are these results also reported in another section?
Row 392: Here you state that it was a limitation to the study that ten surgeons performed the procedures (which I suppose should mean that you think this is too many?), while in another place in the text you criticize a study for only including 3 surgeons. Can you clarify what would be a good way of performing such a study?
Where you comment on health outcomes, and lack of health outcomes assessment, should this be interpreted as there being no health outcomes available in the included studies, or that there are no studies of health outcomes? Sometimes in the results section (and table) you refer to studies that only compared costs, but have you included also all studies that only reported health outcomes?
Do no studies report confidence intervals or other measures of variability between participants, or could that be included in either the main text or the results tables?
Conclusion
Would it be possible to divide this into a methodological discussion (currently lacking), and general discussion about the findings and conclusions? If I understood the instructions from the journal correctly, conclusions should be short and any discussions should be avoided in that section.
Row 433-435: Can you clarify in the text why cost comparison would be appropriate if there is no long-term data, or if you mean that there are only cost studies with long-term results?
Row 450-456: Can you clarify what is meant here? It appears as if you are saying that it would be sufficient to focus on the costs even though you in several places in the text comment on the lack of follow up of clinical/health outcomes, or that you suggest that focus should be on allocation of resources?
What are the limitations of your search strategy and other methodological decisions? For example, how do your search terms relate to those used in other reviews, how were papers assessed for inclusion (by whom and how was it done in practice), and how sure are you of having identified all possible references? Have you for example tried a reference/citation search, or looked for grey literature? How sure are you of your results, with regards to the level of evidence of included studies etc?
References
Please revise your references No 9, 27, and 53. Also, please check correct abbreviations of journal names. You also need to add recent and relevant publication. Most of the references in your current version of the manuscript are old
Figures
For some reason I appear to have all figures and tables in two copies? Please check figure legends, all figures should be possible to read as stand alone.
Figure 1: What is meant by “reference search”?
Tables
Table 2: Could you comment in the text or here on other relevant decisions that were either excluded due to too few studies or are known but have not been studied (since these are presented as the “main” categories of studies identified)?
Tables 3-7: It appears as if you have only graded some of the included studies by level of evidence, is that true?
Tables 3-7: Can you go through how you report different types of information, e.g., cost/QALY is reported for different studies in different columns, and results are sometimes reported for each compared procedure and sometimes as a difference (I would myself prefer to get both).
Table 3: Can you explain the statement that costs for non-operative management was in one case not calculated? Do you mean that it was not reported?