91. Jessica Polka: Preprints, publishing peer reviews, and the joys of pipetting

Jessica Polka is Executive Director of ASAPbio, a non-profit that promotes innovation and transparency in life science publishing. We talk about her work at ASAPbio, how she got into it, preprints,  the many functions of peer review, and much more.

BJKS Podcast is a podcast about neuroscience, psychology, and anything vaguely related, hosted by Benjamin James Kuper-Smith.

Support the show: https://geni.us/bjks-patreon

Timestamps
0:00:00: The Jessica-Polka
0:01:25: What is ASAPbio?
0:03:53: Do we still need to convince people to use preprints in 2024? / Different uses for preprints
0:17:53: Are preprints really that beneficial?
0:24:05: Peer review's many functions and audiences
0:36:36: Do we still need journals?
0:41:27: Why should we publish peer review?
0:54:08: What can we do as individual scientists (other than hope for systemic change)?
0:56:55: How Jessica got involved with ASAPbio, and her day-to-day work
1:08:20: A book or paper more people should read
1:11:13: Something Jessica wishes she'd learnt sooner
1:13:18: Advice for PhD students/postdocs

Podcast links

• Website: https://geni.us/bjks-pod
• Twitter: https://geni.us/bjks-pod-twt

Jessica's links

• Website: https://geni.us/polka-web
• Google Scholar: https://geni.us/polka-scholar
• Twitter: https://geni.us/polka-twt

Ben's links

• Website: https://geni.us/bjks-web
• Google Scholar: https://geni.us/bjks-scholar
• Twitter: https://geni.us/bjks-twt

Links mentioned

The Jessica-Polka: https://www.youtube.com/watch?app=desktop&v=lDdnQytp2eY
(there seem to be many versions)
ASAPbio: https://asapbio.org/
Review Commons: https://www.reviewcommons.org/
Jessica's interview with Everything Hertz: https://everythinghertz.com/51
The Ingelfinger rule: https://en.wikipedia.org/wiki/Ingelfinger_rule
Crowd preprint review: https://asapbio.org/crowd-preprint-review
Peer Community in Registered Reports: https://rr.peercommunityin.org/
cOAlition S: Towards Responsible Publishing: https://www.coalition-s.org/towards-responsible-publishing/
https://scite.ai
Publish your reviews: https://asapbio.org/publishyourreviews
ASAPbio fellows program: https://asapbio.org/fellows

References
Abbott (1884). Flatland: A Romance of Many Dimensions.
Cialdini (1984). Influence: The psychology of persuasion.
Eckmann & Bandrowski (2023). PreprintMatch: A tool for preprint to publication detection shows global inequities in scientific publication. Plos One.
Moran & Lennington (2013). The 12 Week Year: Get more Done in 12 Weeks than Others Do in 12 Months.
Penfold & Polka (2020). Technical and social issues influencing the adoption of preprints in the life sciences. PLoS Genetics.
Polka, Kiley, Konforti, Stern & Vale (2018). Publish peer reviews. Nature.

Chapters

• 0:00 The Jessica-Polka
• 1:25 What is ASAPbio?
• 3:53 Do we still need to convince people to use preprints in 2024? / Different uses for preprints
• 17:53 Are preprints really that beneficial?
• 24:05 Peer review's many functions and audiences
• 36:36 Do we still need journals?
• 41:27 Why should we publish peer review?
• 54:08 What can we do as individual scientists (other than hope for systemic change)?
• 56:55 How Jessica got involved with ASAPbio, and her day-to-day work
• 1:08:20 A book or paper more people should read
• 1:11:13 Something Jessica wishes she'd learnt sooner
• 1:13:18 Advice for PhD students/postdocs

Transcript

[This is an automated transcript that contains many errors]

Benjamin James Kuper-Smith: [00:00:00] By the way, I had a, when I do my intense research for this, uh, for this podcast, I just put your name into Google. That's, that's a lot of the research. Um, and I was, uh, I'm curious, are you familiar with the Jessica Polker? 
 
 

Jessica Polka: Yes, 
 
 

yes, yes,  
 
 

Benjamin James Kuper-Smith: I didn't know how obvious that was. So, 
 
 

Jessica Polka: oh my gosh. Yeah, I, I can't, um, really, I, I can't say that it's my favorite tune or that I've like paid much attention to it, um, over, um, the, yeah, but I know that it exists. 
 
 

So, um, Yeah.  
 
 

Benjamin James Kuper-Smith: I was just curious because I, uh, I thought maybe it's just because I'm in Germany and like the one thing that was shown to me was like a German brass band. So I thought like, Oh, maybe  
 
 

Jessica Polka: Oh, no, no, no. It's here, 
 
 

too. It's here, too, as well. I'm, yeah, I'm sort of glad that Google didn't exist when I was, like, in elementary school, because I'm sure that would have caused even more merciless teasing. Uh, 
 
 

but  
 
 

Benjamin James Kuper-Smith: it sounds like one of those things that can, one of those coincidences that can really make a, make a tough childhood, but yeah,[00:01:00]  
 
 

Jessica Polka: It's fine. I had some very, um, you know, enlightened schoolmates, so it  
 
 

wasn't too bad.  
 
 

Benjamin James Kuper-Smith: good. That's good. Anyway, we're not, uh, you know, not here to talk about various music or songs. Yeah. I mean, I guess we'll be talking mainly about kind of, you know, scientific dissemination and, uh, evaluation assessment, all these kinds of stuff. Um, yeah, maybe, uh, you know, I'll start very classically. 
 
 

Yeah. Yeah. I believe when you work right full time for ASAP. io, what is ASAP. io, what do you do? Uh, maybe, uh, I don't know if it's a company or what exactly you would call it, but the organization in general and you in particular. 
 
 

Jessica Polka: Yeah. ASAPbio is a small nonprofit organization. Uh, we are based in the United States, but we are concerned really with preprint adoption and the use of open peer review throughout the world. So, uh, we began really as a meeting in 2016. When Ron [00:02:00] Vale recruited me and a few other biologists to help him organize a conference about the use of preprints and basic life sciences. So from there, we have grown into a organization that tries to promote community champions in preprint review and And we try to do this through several different kind of theories of change. One is that people begin to change the way that they publish based upon seeing their colleagues and their friends do it. And so what we have tried to do is create a way for people who are willing to change something about how they disseminate their science to find each other, um, gain support, gain the resources and the information that they need in order to have conversations with their coauthors, with their colleagues about posting preprints, about using open feedback and open [00:03:00] peer review. 
 
 

Um, and, you know, we also do a variety of other things. We, you know, of course, assemble these resources. We run some pilots. I think the most significant among them was partnering with Embo to launch Review Commons, which is a journal independent peer review project, but we, I think a major activity continues to be. 
 
 

Convening groups of people together, groups of stakeholders for all the way from people who are very involved in the technical infrastructure of preprint and open peer review publishing to researchers themselves, to people in societies, to journal editors and publishers. Uh, so really through those mechanisms of advocacy, community building, convening, uh, resource building and pilot projects. 
 
 

We are trying to move the needle a little bit in the biology community. 
 
 

Benjamin James Kuper-Smith: You know, before we, before we started recording, we talked a little bit about the interview you did with on Everything Hurts with Dan Quintana and James Heathers. [00:04:00] And, you know, this was something like five years ago and you talked about preprints and how in, especially in biology, they weren't quite as accepted as, well, maybe they weren't psychology, but also just not as accepted as you'd like them to be. 
 
 

In psychology, it feels like. It feels like pretty much everything is pre printed now, although I did, interestingly, have two people who publish, uh, in Nature who I interviewed, like, half a year before, and I didn't know about their pre print, so maybe when people submit to Nature, they don't pre print it, I don't know, and maybe that's some sort of something, Maybe that's an interesting data for it in itself, but I'm curious, like, how do you still need today to convince people in biology that, uh, they should preprint or has that kind of changed a lot in the last few years? 
 
 

Jessica Polka: I think that over the past few years, many more people know. What a preprint is, and they're aware of preprinting and maybe, um, have even posted a preprint, [00:05:00] but I don't think that we're anything close to ubiquitous use of preprints in biology. So, um, I think over this, the past years, there's been a lot of. 
 
 

Really positive change in how preprints are accepted. For example, preprints are now indexed in PubMed Central for works that were funded by NIH. Uh, so I think that that's a, you know, a big help to visibility of these preprints funded by this U S government agency and really helps to legitimize them. But Europe PMC, which is the European mirror of PubMed Central. Indexes of huge swath. I would say almost all of the biomedical preprints and by comparing their volume to the volume of all publications on PubMed. The volume of preprints is still only about 10 percent per month at a maximum. So there's 90 percent of the literature in the life sciences more broadly, the biomedical literature, [00:06:00] is not being preprinted. 
 
 

Now, of course, you can't really make a one to one comparison there because some preprints don't go on to become papers, which is, I think, a good thing. But it speaks to the fact that I think there is more work to be done around Not only, uh, awareness, because I do think that the pandemic and 19 really helped raise awareness of the fact that they exist and people looked at preprints, certainly to, um, maybe, you know, even, even if it was a COVID related preprint, I think people are more aware of preprints now, but I think the main issue is that something that you alluded to where People may still fear that pre printing will harm the perception of novelty of their work when submitting to a journal or it may create problems relating to scooping or being scooped by their colleagues. 
 
 

So I think that these fears still exist and that the institutional recognition for pre printing [00:07:00] or the perception of pre prints hasn't really caught up to really give all researchers the incentives that they need to post them. 
 
 

Benjamin James Kuper-Smith: Yeah. So that's still the case that people, I don't know, maybe it's difficult because I, let's put it this way. Like, I guess I got my bachelor's once the. Um, kind of whole reproducibility movement was really full on strike in psychology. So it's, it's maybe a little bit too hard to imagine, but for me, it feels like, you know, it's, it's, it's basically always been the way that people use preference all the time and no one cares about it. 
 
 

Although, I mean, I did have one talk about the kind of perception of preference. I did have one kind of really weird thing with that genuinely shocked me, uh, in, in peer reviewing an article where I was the reviewer and you know, someone, they, they said like, I don't know, we're not, we are the first people to do X, but they said something like, you know, there's not much that's been done there. 
 
 

And I said like, well, here's three preprints, you know, that did like, you can't really make that claim. Here's three from the last year. [00:08:00] Right. And they actually, then I thought they would just go like, oh yeah, thank you very much. We've included them in the article now, but then they said, no, we don't, we're not going to include preprints. 
 
 

You know, we're not going to cite preprints in our final manuscript. And the reason they gave was because we want to respect the work that reviewers put into reviewing period articles, something really weird, but, um, I'm curious, have, is that a common thing that people just don't want to cite preprints at all? 
 
 

Because I still see them as That's brilliant. I mean, sorry. Also, my argument was saying like, you are a peer, review it. Now it's peer, like, you can make, you're two authors. You can just make a judgment of whether this paper is worth to be included or not. Then it's peer reviewed. Right? I mean, it's, it's also from your field. 
 
 

Uh, it's not like this is like some completely out of field thing. Um, but is that still, is that common? I don't know. I don't, I don't have that much experience with, with peer review and preprints and that kind of stuff. 
 
 

Jessica Polka: I definitely think that it is. I, to my knowledge, I'm not sure of any work looking at whether [00:09:00] Preprints are underscited systemically because of things like this, but certainly we hear complaints from researchers also that less in the last couple of years, but that journals are not allowing them to include citations or requesting that they remove citations to preprints. Which is, in my view, kind of outrageous. It's, um, uh, plagiarism is a very strong word, but to take the idea that there may have been information that is useful in a reference, and then to force authors to remove that reference, or to neglect to include a reference that is actually useful to you, that seems like a Pretty bad way to represent and share scholarship But what you're describing of researchers just simply not wanting to look at preprints seems also equally strange to me Would you not go to a conference and listen to a talk or go to a poster? 
 
 

That's Representing something unpublished. [00:10:00] It's confusing. It's I think it's an example of how Our current system for information management in much of scholarship, which is to sort of rely on peer reviewers and editors. To create a absolute, unidimensional, really binary metric of trust, if something is published and peer reviewed, then it's kind of accepted that it's trustworthy, which, of course, we all know to be based upon the incidence of retractions, um, to be not necessarily a great strategy. But I think that it's an example of a time when that way of thinking has to be challenged a little bit. And so, it's not surprising to me that some of these, uh, growing pains exist. 
 
 

Benjamin James Kuper-Smith: Yeah, I mean, also, I don't know how, how well you can comment on this, because I guess I'm, I'm curious, like what the differences between different fields, because it seems to me that from what I hear, mathematics has a lot of stuff that is just on [00:11:00] archive. And that's kind of it. So I don't know whether that's true. 
 
 

It's just a rumor. I don't know. But I definitely know that, for example, in, I mean, in economics, they have a very different, you know, which is a field I sometimes read papers from, they have a very kind of different field, because they have these papers that also take years to come out. So to, allow PhD students and kind of early career researchers to have something to show, they have working papers. 
 
 

So they have these like kind of working papers that they put out there that are basically preprints. It has a different, I don't know, I don't know the economics what exactly it has, seems to have some, not exactly the same status as a, as a preprint. But I'm curious, like, is there, is, is, is biology like one of the, one of the later sciences to accept this kind of thing? 
 
 

Or is my view just skewed because I happen to know, you know, the stuff that I know happens to be. Happened to be fields that are very heavily used them already. 
 
 

Jessica Polka: Yeah, absolutely. I do think that biology is late to the game, although there was a really interesting experiment in the [00:12:00] 1960s, where the NIH ran an experiment on called information exchange groups where they literally had people mail in papers, they photocopied them, then mailed them back out. Um, so I, I do think that there's, it's, it's not necessarily the biologists will never, we're never going to move in this direction, but I do think that certain practices and policies, namely the Angle Finger Rule at the New England Journal of Medicine really chilled the willingness to share information in a format that is approaching a preprint for several decades. 
 
 

Benjamin James Kuper-Smith: Sorry, what is that rule? 
 
 

Jessica Polka: The Inglefinger rule was instituted by an editor of the same name who basically suggested that medical research should only be published once, that in order to keep the literature comprehensible and clean, there should not be duplicate publication. And so this rule, which has become kind of assumed [00:13:00] practice in much of at least the biomedical literature, has I, I think precluded preprints from being accepted because I think there's a little bit of an assumption that that it is duplicate publication. I think that there's a question as to whether you characterize preprints as duplicate publication or as more akin to a conference presentation where people are sharing. Um, I think once you press publishers and editors, I don't think that anybody wants to prevent researchers from. Talking to one another. 
 
 

The question is really how similar the preprint is to the work that's going to appear in the journal eventually. 
 
 

Benjamin James Kuper-Smith: In a way, I like the rule to like keep everything, you know, to keep the number of publications from exploding so much, but, I mean, I see where it comes from, but the weird thing is because, you know, the, the, the, the, I mean, the preprint is obviously going to differ from the final product, but at least the way, I don't know, maybe actually, maybe this is a question, how people use [00:14:00] preprints, because I actually use them as, here's the thing that I wanted to write, and then, um, it gets changed afterwards, uh, through peer review. 
 
 

Uh, you know, often you learn stuff because, you know, obviously peer reviewers point things out, but for me, the, the preprint is supposed to be basically an article. Is, is that actually how it's usually used or are there uses for preprints? I actually never considered that there, people might do things differently than I do. 
 
 

Yeah. 
 
 

Jessica Polka: Yeah, I do think that the majority in various surveys, at least in biomedicine, I think people are typically posting the preprint around the time of submission to a journal. But that doesn't mean that people can't post preprints that they never intend to submit to a journal. Or they can't use preprints for sharing of information that is not going to be typically welcomed by the journal process. For example, earlier this year, we ran a competition. For preprints containing negative data, uh, so can preprints be a [00:15:00] vehicle for sharing negative results that change the equation a little bit? I think, you know, perhaps for a researcher, if you're sitting on some data that are not going to make a huge splash, it may not be appealing to go through. 
 
 

Um, I think it's a process of trying to convince a journal it's worth publishing, getting peer reviewed, doing the revisions, whereas I do think it's important for the community to just kind of get it out there. Do the incentives line up to encourage everyone to use preprints in this fashion right now? 
 
 

No, but I do think that it's something that should be done more in a more widespread way. 
 
 

Benjamin James Kuper-Smith: Yeah, that's a fair point. I mean, yeah, I have an article right now that's just taking me for ages to get published. And at some point you go like, is it worth the effort? You know, it's already a preprint, but like, you know, 
 
 

Jessica Polka: Yeah, not only that, but I think, um, being able to put in the effort is a huge privilege in a sense, there was a really interesting preprint that came out earlier this year, I believe, called preprint match, which was describing a tool [00:16:00] to identify matches between journal articles and preprints, and they used it to identify additional Preprint journal article links beyond what is simply reported in preprint servers or Crossref. And they use this information to look at country specific rates of conversion of a preprint into a journal article. So in wealthy countries, um, in Europe and in North America, for example, something like 60 or 70 percent of preprints eventually become journal articles. But in low and middle income countries, That number is closer to 40%, and there's also a relationship between whether the researchers have external funding or not. 
 
 

And in those contexts, the time window between pre printing and the immersion of, um, this final article, or the emergence of that final article, Is a shorter time [00:17:00] window and there's more changes in author list between the two. So it's almost as though in order to take a paper from submission that a journal all the way to publication at a journal, you need to have people staying in the research group long enough to complete the experiments to respond to reviewers. 
 
 

You probably need to have some funding to, you know, conduct the additional work, um, to say nothing of publication charges or, um, anything that might be associated with that, any costs associated with publishing. So, right. I think that preprints from that lens are really allowing more work to emerge than would if we had only the traditional 
 
 

Benjamin James Kuper-Smith: really interesting. I never thought about that. 
 
 

Jessica Polka: Right. 
 
 

Benjamin James Kuper-Smith: I mean, I guess that's the whole, you know, benefit of the internet, right? So that basically anyone with a connection can share things and you don't need, you know, access to X or Y or all these other things. I have a, I have a brief question about, uh, about preprints in the sense that one of the advantages of preprints that's, that's often mentioned as one of the [00:18:00] main ones is that you can, I guess there's like two main advantages of preprints that I'd like to question a little bit and see kind of, uh, what you think about, uh, my potential objections. 
 
 

I mean, one is kind of that, People say often that preprints are great because you can get feedback before you publish. You're right, you put your preprint out there, people comment on it there, and then you go like, oh thank you very much, here's an error, I fixed it, you submit it, and then it's, you know, the final product's better. 
 
 

I've never had anyone really comment usefully on any of my free friends. I mean, part of that is because probably me and the lab I was in and the people I know are not huge on social media. So like I can mail, I've like, I've mailed people my basically manuscript in that sense. And then they've given me useful feedback. 
 
 

But like, I don't have the kind of social media following that loads of people will necessarily read it after that and actually comment on it. So for me, it feels like it's a kind of one of those uses that. Might happen, but not really, at least not in my experience. I don't know. 
 
 

Jessica Polka: Yeah, that's a, it's really interesting that you say that because I completely agree with you [00:19:00] that in a perfect world, everybody would be getting really constructive feedback on their preprint. They would be making connections with new colleagues that weren't aware of their work. They would be starting collaborations. 
 
 

But I do think that this vision is limited by a couple of different things. The first being that. Right now, attention on preprints, I don't think we've really figured out how to give the right preprints our attention. As you mentioned, you mentioned social media. There's a lot of use of the network formerly known as Twitter and other social media tools to share preprints. 
 
 

And I think that that's It's fantastic, but it can also create this deluge of information, and I have spoken to many researchers who mention using the table of contents of journals as a way to identify work that they want to read, and, you know, clearly you can [00:20:00] get a table of contents for a subject matter collection at a preprint server, for example, but it's going to be enormous, it's not going to be filtered. Clearly, I think that there's a lot of work being done in tools to recommend papers, in search for specific keywords, following certain authors. I think there's a lot to be done, um, in this space. But the more, uh, significant barrier in my view to public commenting on preprints, because, you know, as you mentioned, it's just not that common on bioRxiv, for example, I think between 5 and 10 percent of preprints have a public comment. But according to a survey done by bio archive, the number of authors who received feedback privately is much higher than that. I think something, um, you know, several times higher, certainly, and the feedback can come in several ways. And I do think that people are receiving those benefits, but the benefits of the community are really limited because it's not being [00:21:00] shared publicly where others can see it, where it's effectively changing norms. about sharing opinions about papers publicly. At ASAPbio, we've tried a couple of different experiments to help reduce the energy barrier to posting a public comment, because obviously it's completely understandable that especially early career researchers might feel some hesitance in posting a comment publicly, especially if it creates some kind of, um. Critical comment that creates, uh, an awkward situation for the authors and the commenters. So What we've done is launched an initiative called Crowd Preprint Review, which is based off of a type of review initiated by the synthetic chemistry journal, Sinlit, in which a pool of authors are, excuse me, of reviewers are commenting on a paper. All together, and then those comments get synthesized into a single review that gets posted [00:22:00] publicly, but the synthesis enables the individual reviewers to be anonymized. It enables them to kind of comment and cross comment on one another to correct or to provide feedback on the commentary so that it becomes a bit more robust. I think that for researchers right now. There's so many invitations to peer review things getting sent around, though, perhaps not necessarily to the people who have time to actually do it. And as a result, it can be kind of burdensome to imagine leaving an unsolicited review or an unsolicited comment on a preprint. 
 
 

So I think that there's a variety of initiatives now that are trying to. use some kind of coordinator, a community coordinator, just to kind of stimulate this type of feedback and review. And I think that's what needs to happen in order to enable feedback at a large scale. 
 
 

Benjamin James Kuper-Smith: Has that already happened a few times? Was that still in the, in the making, that scheme? 
 
 

Jessica Polka: Yeah, so there's, so crowd review, um, we've run it for [00:23:00] three years now with various different permutations, but I think the most significant source of preprint review is actually from the journal eLife, which has now shifted to a model in which rather than providing accept reject decision, they are. Posting manuscripts that have been reviewed along with the reviewer's comments and an editorial summary, which allows the authors to go back and make a revision, uh, you know, get the editorial process, um, get another editorial evaluation. But, um, I think that that is really addressing a lot of the volume that we're seeing. 
 
 

So we have a. Chart of the volume of preprint reviews per month, and I would say that this has increased over the past few years. If you give me a minute, I can tell you the numbers exactly, but I will say that they have risen from almost non existent [00:24:00] about six years ago into I believe the hundreds per month now. 
 
 

Benjamin James Kuper-Smith: I mean, I guess kind of one topic that's been in the, in the background kind of popping in and out of our conversation is the idea of peer review as the decision whether to accept the journal or not. Whereas what we're discussing right now is kind of peer review, uh, uh, yeah, peer review kind of just to help people, you know, basically just to say like, here, here's what I like and what I didn't like about your article. 
 
 

Could you maybe outline the, kind of the current state of the idea of, you know, peer review with, separating peer review from an acceptance to publish in a journal or not? 
 
 

Jessica Polka: Yeah, absolutely. Peer review does have multiple functions, and it has multiple audiences. As it's used right now, it provides feedback to the authors to help them improve their manuscript. And it also provides some context to the editors and maybe even advice to the editors on whether or not to accept a paper [00:25:00] for a journal. But Peer Review could also have, and does, in cases where journals actually publish the Peer Review reports, a third audience, which is readers of the paper. And I think here it helps to contextualize the paper and maybe provide a little bit of information about the strengths, um, other considerations, other interpretations. This is something that is not really emphasized, of course, in the journal review process, but I do think it's a strength that can be played up as we experiment with these different forms of, of preprint review. Um, I, I think that taking the. The concept of journal acceptance out of the review process changes the way that reviewers act. I don't think that there's been very rigorous, um, like semantic analysis of the difference between review reports for a journal agnostic review service [00:26:00] versus a traditional journal. But what we do have is a survey conducted by review commons in the early days, uh, after its launch, where authors were asked, if you compare the experience of getting reviews through Review Commons, which are journal agnostic. 
 
 

So basically what happens is that the authors will submit a manuscript to Review Commons, a editor at EMBO will assign reviewers, but the reviewers are asked to focus on the science and not evaluate the paper for a specific journal. They send the reviews back to the author and the author can then take that package of a reviewed paper, which now becomes a reviewed preprint. But also submitted to affiliate journals who. Agree not to restart the evaluation process. So basically what it is is a single round of review that is not going to result in like an editorial decision directly. And so when authors are asked, how do [00:27:00] you find this process? compared to the more traditional journal process. The majority of, of the respondents evaluated the review commons experience as more constructive and more I would say kind of pleasant and collegial. And I think that's certainly hopeful for. we're addressing some of these issues that we have where, you know, you have lots of editorial cartoons. You can probably imagine in your head right now, peer review as like a, a gauntlet where reviewers are swinging maces and axes at this white coated scientist trying to get to the goal of publication. But it doesn't necessarily have to be. That way, peer review, when it's removed from gatekeeping, I think still has an important role in informing readers about the context of a paper. And I certainly. There's papers that have, that are very misleading or that can be damaging to public health, and that's a big reason why I think that [00:28:00] some of the fields relating more directly to clinical science and medicine are more hesitant to embrace pre printing because the risks, of course, are much higher. But just to say that even if you don't have journal gatekeeping, I think reviewers still have a way to express to readers of the paper that there might be other ways to interpret the information presented. 
 
 

Benjamin James Kuper-Smith: Yeah, it's interesting when you talk about the review comments, I mean, it sounds very similar to a peer review in which I talked a little bit about with, um, Chris Chambers in the context of register reports where they have, you know, You, you submit your reg report and then you have people evaluate it and then you, I think that then you get an accept or reject, but independent of, there's like 10 journals or whatever, who will just take it if it's, you know, if it fits within the topic. 
 
 

It sounds very similar. Are there, are there any kind of interesting differences that maybe highlight different nuances or? 
 
 

Jessica Polka: Yeah, so I registered reports. Um, as I understand [00:29:00] them, the context that you're mentioning, Enable researchers to get peer reviewed for the methods independent of the results. Whereas in Review Commons, the, um, and a registered report, as I understand it, is conducted, except in the case where you mentioned with peer community and, um, would be conducted by a specific journal. Whereas I think Review Commons is similar to peer community in, in that it is a journal independent review process, but typically by the time. That a manuscript gets to review commons because the field is really basic biomedical sciences, lots of cell biology, for example, molecular biology. You know, for that reason, it does not resemble a registered report in the sense that it's really like a complete manuscript. 
 
 

But I love the idea of pre registration. And I think that it's kind of a puzzle, though, to think about how to implement pre registration in the kind of science that I was doing in my PhD and postdoc where The time [00:30:00] window of experimental iteration is really high. Like you can have an idea for an experiment, just go into the lab, mix some liquids together and have the experiment done by the end of the day. 
 
 

So it's, um, you know, it's, it's challenging to think about that. Um, although I have heard Brian Nozek from Center for Open Science talk about the idea of integrating registration with a lab notebook. So, you know, the idea that you're kind of registering experiments on a daily basis as you're planning them out. 
 
 

And that to me makes a lot of sense, but it also really upends in a much more fundamental way, uh, the notions of publishing, even compared to what we're talking about now, 
 
 

Benjamin James Kuper-Smith: Yeah, I mean, another, again, asking a little bit about the purpose of preprints, another kind of question I had was kind of what exactly the purpose of a preprint is, if in many cases it's, it feels like, I mean, it, yeah, it's, it's not like a full publication and often it's only fully accepted once it is a publication. 
 
 

[00:31:00] But, uh, yeah, so maybe what are kind of some, some of the purposes of preprints if it's often not as accepted as an actual publication. 
 
 

Jessica Polka: I think there's two different ways to look at this. 1 is that you're posting a preprint on average, according to PubMed Europe, PubMed Central, or it's about 4 to 5 months prior to the journal publication. So. You're posting a preprint some months ahead of when the work is going to get out there. We know from various studies comparing preprints to final published versions, that the majority of preprints more or less resemble the journal article. 
 
 

Typically the sample size might increase. Some content might get added or removed, but generally the The main conclusions of the paper stand, although that's not of course the case for every single preprint and there's important conversations that we had around that. But nevertheless, I think, you know, one argument is that you post the preprint and in that time you help people [00:32:00] in your field gain some knowledge and understanding. begin impacting their own conception of the problem that they're working on and Accelerate the process of discovery and save time for early researchers and everyone else another way of looking at pre printing is that by putting the dissemination first and layering peer review and curation and reputation on top of that You can create a new form of publishing that breaks apart some of the problematic associations that we have right now with sharing work in journals that Create prestige signals. Uh, so I think in some ways the process of right now, dissemination in a traditional journal is wrapped up really tightly with, of course, peer review, which is an important service that journals provide, but also with the assignment of [00:33:00] a certain presumed quality, uh, marker to, to the paper. So even though we've had lots of initiatives like Dora that have made it distasteful in some circles to mention the impact factor, I think that everyone has a reaction to looking at the journal title as a way of understanding what the authors were able and or willing to go through in order to get the paper out. So on some levels, I think it is important that there are. I don't think that it's pragmatic for a human brain to try to comprehend the totality of literature that is being produced on a daily basis. It's simply impossible. Certainly one can do that for your own narrow field, but what happens when you step outside that field and you need to, um, Evaluate a potential candidate to be hired or evaluate someone for [00:34:00] tenure or, um, you know, as you, as you go up into this administrative ladder in institutions and it becomes, um, much more challenging to understand science. 
 
 

And so journal title is really, I think, the shorthand that has been developed as, as a metric to. Signify quality, and I think it's a pretty bad way to do that for a number of reasons. First of all, it's collapsing all of the possible quality indicators or significance indicators or rigor indicators into a single. Word. Um, the evaluation process is done by a relatively small number of people, which becomes more and more challenging. The more interdisciplinary the science is, and it also happens very inefficiently. It happens. in parallel. So not in parallel, but in series. So for example, you get rejected at one paper that, uh, [00:35:00] excuse me, you get rejected at one journal, you take your paper to the next journal, you go through another process. 
 
 

And, um, it, to some extent, there's, there's a endurance contest. Aspect, which I think 
 
 

is really made clear by that paper that I mentioned earlier, the preprint match paper. So I think that preprints offer an opportunity to rethink publishing. And what I think is great about them is that even though they have this very aspirational visionary potential, They are also compatible with journals, and so it allows people to engage with, you know, a taste of open science, if you will, in a way that is sensitive to their current needs and their current willingness to do so. 
 
 

So, for example, somebody may pre print their paper and submit it to a traditional journal, someone who is feeling, um, You know, perhaps that their context allows it might preprint and [00:36:00] use one of these preprint review initiatives, especially now that there are some funders and institutions that are recognizing reviewed preprints, um, as, as kind of evidence of productivity. Um, so there's, there's all kinds of different ways to envision them, but, you know, from the perspective of thinking longterm, I do think that the opportunity to revisit how we think about. Peer review, what peer review is, the functions of peer review, and how we think about quality and rigor and significance and all of these abstract things. That's what really excites me the most about preprints. 
 
 

Benjamin James Kuper-Smith: Do you still think we need journals? Kind of. What's your, uh, it's a big question in a way,  
 
 

Jessica Polka: Yeah, I  
 
 

Benjamin James Kuper-Smith: I'm, curious, kind of what you and maybe a sub bio, whether you have a preferred future model of scientific dissemination that kind of ties together these quality indicators and, you know, some of them maybe necessary, uh, um, curation, um, work [00:37:00] that journals do. 
 
 

Do you have a. Particular suggestion there. 
 
 

Jessica Polka: Well, I think the journals are doing really important work in providing peer review and in curation, and I think that that work needs to be done. In some fashion, I do question whether the current journal publishing workflow is the best way to do that work. I am intrigued by the latest proposal from Coalition S, um, called Towards Responsible Publishing, in which a variety of principles for publishing are laid out. including that authors kind of control the dissemination of their work. So in other words, that they are the ones who decide to publish their paper, to post their paper, and that the peer review process is transparent and open and kind of putting those things together. You do end up with a. Preprint and then [00:38:00] review model in which authors post their preprints, and then various quality control processes can be layered on top of that. And some of these might resemble traditional peer review where an editor invites reviewers, expert reviewers, maybe where the editor writes a summary, kind of like we see happening in eLife right now. Others might resemble more informal commenting, so there's so many different experiments happening now in pre print review. You know, I think there's been interesting experiments in badging by the American Society for Cell Biology, Plotted, which is like literally just a button that you click that indicates if you find a paper interesting, you 
 
 

log into it with your ORCID. 
 
 

You know, there's there's Yeah, 
 
 

it was like AI driven review or, you know, automated checks for compliance with, um, with checklists, with data availability. Uh, so the concept of [00:39:00] evaluation right now is, you know, everything that is kind of collapsed into this single trusted stamp of approval of a journal title can actually be. Made transparent and made all of this rich information can, I think, help readers understand on a much more dynamic level. How does this paper, how is this paper viewed? 
 
 

Um, can I trust, uh, the information here? How does it relate to other work in the field? And, and I think that all of that could be done in a much more dynamic and ongoing way. Uh, especially with some emerging technologies, you know, for example, I'm thinking about site.ai, which looks at the context of how a paper is cited by other work, and is that, is it a confirming or, uh, does it refute it, et cetera. 
 
 

So I I think that there's a lot of opportunity to evaluate papers in different ways and to also create. Um, [00:40:00] lists curation, both for readers and for people who are thinking about evaluation in a way that's a lot more nuanced, but in order to do that, um, I think we need to have some alternative to the traditional journal publishing system, um, or at least like the permission, um, to kind of layer some of these experiments and evaluation alongside the existing literature. 
 
 

Benjamin James Kuper-Smith: Tony, when you mentioned plaudit, I was like, I was like, definitely heard that and then I remembered, oh yeah, that thing on the top of SciArchive that no one ever uses. I mean, not to shit on the people who did plaudit, I'm sorry, but, uh, 
 
 

Jessica Polka: Yeah,  
 
 

Benjamin James Kuper-Smith: I don't know, it's like,  
 
 

Jessica Polka: no, it's, I mean, I, you're, you're right. It's like there's a lot of these adoption is not, um, it's challenging to get adoption started because I think there's a big chicken and egg problem here where no one, if there's, you don't have a huge user base, it's going to be hard to get meaningful indicators. 
 
 

And if there's no meaningful indicators, then [00:41:00] use of that. Modality is not going to be very high, which is in turn, does not promote using so user base. So I do think that there's an important role for institutions and funders to actually step in and, yeah, be, I would say, a bit bold about what they want to see in terms of evaluation, because I think that could really help, uh, put some gas behind some of these experiments. 
 
 

Benjamin James Kuper-Smith: Uh, why should we, why should we publish peer review? 
 
 

Jessica Polka: Peer review is such an important aspect of scholarship. It's really making or breaking people's career. It's kind of what's providing the fuel for our current evaluation system and our trust in science. But the fact that we most peer review now is seen by only a few authors, uh, excuse me, a few [00:42:00] readers. 
 
 

It's very wasteful in my opinion. It also, when this review is not shared publicly, it. It kind of aids in again, this collapse we've been talking about of this, like, of overcompressing all of the information in the review and evaluation process, just down to journal title. So why, why do we want to lose that information? 
 
 

Um, it can provide insights onto. The strengths and the weaknesses of the paper. It can help students who have many times never seen a peer review report 
 
 

until they get their own back on their own paper. Um, and 
 
 

Benjamin James Kuper-Smith: they get a rough awakening. 
 
 

Jessica Polka: Yeah, exactly. Right. And, and I think there's a lot of heterogeneity in peer review reports. 
 
 

And I think that bringing them out into the public and into the open would perhaps inspire people to reconsider, you know, how, how they do them, um, that best [00:43:00] practices could be better identified. And frankly, I think that it's a. step, though not the entire solution toward rewarding and recognizing reviewers for their efforts. Right now, people's motivation for reviewing my understanding from the surveys I've seen team seems to be around helping the field altruism, although probably relationships with editors are not insignificant. As well, , right? 
 
 

But I think that, that this is an important work and, and, um, there's a reviewing crisis. 
 
 

There's not enough, uh, perhaps acknowledgement of, of the scholarship. And, and I think it would be very healthy to do that. And furthermore, I think that that hiding all of this review enables behavior that is perhaps not the most collegial and not the most. Um, constructive. I, I think that often talking to people when they say, you know, I, I write the report as [00:44:00] though it's going to be posted or as though I'm going to sign it, even if, um, even if I'm not. 
 
 

And I should clarify here that I, I'm not arguing that everyone should sign their reviews. I do think that there's a lot of complications around that. I think what's most important is that the content of the review report actually makes it out there rather than the identity of the reviewer. But all that is to say that, um, I think that if you're writing it for the public audience, it's it's a little bit less of an opportunity to, you know, one needs to reconsider how they're phrasing things, how they're saying things. And I think that would probably, uh, go a long way to making the whole publishing experience, uh, more, more constructive. 
 
 

Benjamin James Kuper-Smith: Okay. Uh, brief question there. Can actually I, as a peer reviewer, can I just publish my peer reviews of, of things? Or what's the kind of I don't want to say legal, allow it, but like, I actually don't know, like, what the, feels like I just write something and then it, you know, it's [00:45:00] given to the journal, but kind of, am I allowed to do that myself or does the journal have to have a policy? 
 
 

Well, yeah. How does that work? 
 
 

Jessica Polka: Absolutely. Um, there's a lot of confusion around this, but reviewers are authors of their peer review reports. And as such, they own the copyright of those reports. And I think that there's a journal confidentiality. Um, norms that reviewers are kind of held to, to not reveal information about a manuscript that is under review. But where there is a pre print, I think it's very constructive for reviewers to post the content of their reviews. And we've had a campaign, uh, called Publisher Reviews to, to encourage this behavior because I do think that it would. Really have all the positive effects we just discussed and reaffirm that reviewers are authors of [00:46:00] this scholarship. 
 
 

And by posting it on preprints, I think there's also an opportunity to help stimulate some of this public discussion that I think could be so powerful. 
 
 

Benjamin James Kuper-Smith: I mean, if peer review comments are published, but anonymous, doesn't that kind of defeat some of the purposes because you can still be an arsehole without anyone knowing it was you? I mean, just some of the, you know. The, the being more kind of constructive. I mean, if it says, if your name is associated to it, then basically it's an anonymous, like comment board on the internet, right. 
 
 

Apart from the editor  
 
 

Jessica Polka: Yeah, I I hear you. Um, I, I would like to, perhaps I'm being naive, but you know, I, I would hope that people would still, you know, consider the implications, uh, for the, for the authors regardless. But I think that there's a couple of different ways to potentially create systems where the identity of the reviewer is known to certain parties who [00:47:00] may. need to have this information. For example, a institution or editors who might be interested in looking at the review work of particular individuals. For example, there could be a system where there is It's mediated anonymity where the review is posted in an anonymous way, but the identity is tracked by the system posting the comment. For example, pre review has a mechanism by which early career researchers can post under a pseudonym and then later, if they wish, attach their name to the comments so can kind of unmask themselves later. And yeah, I definitely, I, you know, I hear what you're saying, but I think that there's also, we need to balance the. potential benefits of people signing the reviews against the very real fear of retribution that I think especially early career researchers or people who [00:48:00] feel or are underrepresented or marginalized in academia. Um, and I, I don't, I think that there would be, it would be challenging if we excluded more of those people who I think we should be doing more of the review than they currently are, um, from, from being part of the conversation. 
 
 

Benjamin James Kuper-Smith: should you, if, if, I mean, I'm wondering like, uh, should reviews be, uh, should you be allowed to, it's just interesting, like if once it's public, it just really kind of changes in some sense what it is. I mean, obviously if it's. You know, the standard model right now It does have that kind of finality of decision because you know You influence the editor and that kind of stuff and but it seems to me that it's kind of there's a different kind of finality To it when you know, once a peer reviewed comment is out then it's public Especially even you know, I mean you could you could be just wrong because you don't understand something Or maybe even worse you could [00:49:00] be wrong because you understand something and then you realize a year later Oops, I didn't understand that. 
 
 

You want to correct it. So should you be able to like comment on your own reviews or how do you deal with kind of, you know, bad peer review? That's then, you know, I mean, that's one of the classic fears, right? That you, you kind of attach this incorrect and negative potential perspective to an article forever. 
 
 

Jessica Polka: Yeah. And it absolutely. I mean, that, that is happening now in the current journal system, but it's not necessarily being. Revealed. So you may have inadvertently, you know, for example, by, you know, in your, in the case you present, let's say that there's a review, a viewer who misunderstands something about the paper, um, you know, their review could certainly impact the, the outcome of that paper, whether that paper is published or not. But I think the benefit of bringing the review into the open is that there is [00:50:00] this, um, I guess what we would call, uh, you know, in the United States, like bystander, uh, training where, you know, you, as, as a third party who is able to view something happening, you have an opportunity to kind of step in and say, I really agree with this. 
 
 

I disagree with this. Here's a different way of looking at it. So by broadening that pool of expertise, I think that we collectively could potentially get to more accurate assessments or more fair assessments faster. Now, I think you also touched on this idea, well, if I publish something, then I'm going to be wrong, you know, on the internet and like, yeah, that is a horrifying idea. 
 
 

Like I completely agree with you. Um, but 
 
 

Benjamin James Kuper-Smith: I mean, it's basically like the, the peer review equivalent of seeing your old Facebook posts when you were 15 or something,  
 
 

Jessica Polka: yeah, no, yeah, exactly. I, right. [00:51:00] It's. It's, it, I mean, it, to some extent, like there's, that is something that we as a culture are probably, you know, running into in a variety of different ways, both personally and professionally, like these ghosts of the past coming out. But I, I do think that there's something really sensitive here about. 
 
 

Right. Yeah. I mean, no, I think, you know, there's, there's a lot of questions about, and I think we need and a lot of feedback from people, right? But, I mean, yeah, I think it's really important to us as a community to be able to ask, like why would you have a document that says, you know, anything specific about the persons or the answers to your questions, and then tell it how you're feeling about your questions and how you feel about your concerns. Disputed, but we don't really have the granularity to say, like, exactly this is not necessarily something that everyone believes anymore. Or, you know, it's, it's, um, we're moving from a super oversimplified way of validating or certifying information into an environment where there's a lot more [00:52:00] openness and a lot more. 
 
 

And so I think that we can learn something from social networks. Like, so for example, Twitter has inserted this context, these like context cards under tweets that are controversial, where I believe that they're actually written by other community members who want to provide important context. And so by kind of highlighting Some of that information and presenting it to people alongside. 
 
 

I think that that's one way that we can help to create. A more nuanced approximation of the truth. Um, so, you know, I think it in, in life sciences or in biology right now, there's papers, of course, that have been retracted, but other papers that I think, you know, they're not necessarily wrong, but perhaps the field has changed the way that they think about things. 
 
 

And, [00:53:00] um, that is not something that I think implies that the work was done without. care or rigor, but it would benefit everyone if we were able to have more context, uh, about how thinking in the field has changed over time. 
 
 

Benjamin James Kuper-Smith: Yeah, I guess, especially for outsiders of the field, right, I mean, I guess that happens quite often that you're like, oh, here's this like really highly cited paper from this one field and then you find out, oh, they've like, they've moved on. That's like, you  
 
 

Jessica Polka: exactly, exactly.  
 
 

Benjamin James Kuper-Smith: outdated 10 years ago here. Yeah, 
 
 

Jessica Polka: would, you would know that, like, just by speaking to someone, if you had the right connections. 
 
 

Or, you know, it's, it's the information, yeah. 
 
 

exactly, the information is inside many people's brains, but it's just not really expressed and shared in, in a way that enables everyone to have access to that information. 
 
 

Yeah. 
 
 

Benjamin James Kuper-Smith: like a bunch of different ways in which, [00:54:00] you know, some things work, some things don't work, and this kind of stuff, and how we might want to improve things, or what aspects we might want to incorporate in some form. Uh, one question I kind of have is, is there anything we can do as actual, as like individual scientists? 
 
 

Or I mean, we can maybe, you know, maybe do something like you're doing and do it full time. Um, but for people who, you know, don't want to stay scientists, like, is there any kind of, is there anything you can actually do about this kind of stuff yourself? Or is it kind of just you hope that the norms change? 
 
 

Jessica Polka: Yeah, absolutely. Um, I think that there's a variety of different actions that individual researchers could take, ranging from the mild to the much more intense. So for example, I think one simple thing to do is to simply watch preprints and look for peer review, look for Comments or feedback, for example, in certain preprint servers on bio archive, there's not only the comments section, but also [00:55:00] a panel where you can pull out and link out to different forms of review. So, um, that might not be something that is front and center for on everyone's mind, but I think being aware, looking for those, bringing them up in conversation, I think a huge vehicle for an opportunity. for normalizing the use of preprints and actually highlighting the benefits of preprints is in journal clubs. So, at least when I went to grad school, journal clubs were a way to demonstrate how smart you are by trashing a published paper, and then nothing is done with the discussion. But, if instead, A group of grad students or really lab members or whoever were to discuss a recent preprint in their field, they could write up their comments and send those comments to the author or even better post them publicly on the preprint to help influence that work and to Grow [00:56:00] connections within their field. Um, I think that it's just a much more constructive use of time, in my opinion. Um, yeah. And I think that, um, for someone who is looking for more is that bio, we run a fellows program, which is essentially like a six month program where. Early career researchers will get some, uh, exposure to kind of the research about preprints, the meta research about preprints, um, some opportunity to practice kind of advocacy and discussing preprints. Um, and also work on projects relating to do preprints too. So that is something that I think might be an opportunity for somebody who's looking to kind of be more of a champion locally in their local environment, their institution or their society and try to, um, get some, get some change happening. 
 
 

Benjamin James Kuper-Smith: Okay. You know, I asked my last question for people who kind of want to Stay [00:57:00] in kind of traditional academia, science, that kind of stuff. Uh, what if basically how you've, you've been, I mean, taking a, you know, much more, not an extreme opposition, but you know, you, you work on this full time. I'm curious kind of how, how that's happened in your particular case. 
 
 

And maybe also, uh, what, what kind of other jobs like this are out there. Was ASAPiO the first job like this that you did or did you have something else before and kind of how did you get there? 
 
 

Jessica Polka: This is, ASAPbio is something that I started working on during my postdoc, but prior to this, I was involved in Uh, another organization and actually a couple other organizations relating to kind of scientific workforce, um, looking at incentives in the scientific enterprise and that, you know, really began during my postdoc kind of looking around, you know, I was at [00:58:00] a, um, an institution where, you know, I felt like there was a lot of competition among researchers. 
 
 

There was, there was a sense of urgency and kind of. And, you know, it's a sense that one had to optimize your science for publication, not necessarily for doing this, the science, if that makes sense. 
 
 

Um, and you know, if you think about it, like, what are we being, what are we being rewarded for doing? Um, what is all of the, the kind of funding and institutional structures supporting? What kind of behaviors are, are they? And I don't, I'm not sure that they're really the behaviors that we most want to see among researchers, which I would argue include things like collaboration, being, you know, a very supportive mentor. Sharing, openness, all those things, even rigor, I would argue, is not really immediately rewarded in the same way that telling a good story about a [00:59:00] selection of your work that happens to fit together, um, that the rest of which will be buried in like a file drawer, you know, I think that there's, there's a variety of different ways Kind of challenges that became more obvious to me once I emerged from the cocoon of a PhD program into seeing, you know, more how, how people were competing for academic jobs and, and funding. So I originally started working, um, with some Some postdocs in the Boston area, because there's a huge concentration of postdocs around Boston. Um, and we, um, put together a meeting about, um, the, the research enterprise, which, you know, I think relates to ideas of how these incentives impact a population that is experiencing a high level of competition, which is. Possibly a result of the fact that the biomedical research workforce relies on training as its primary labor force. Um, it's not as [01:00:00] though it's It's, it's, uh, there's a professionalized labor source. The labor source is really trainees who are, you know, expected to go out and leave the system and, and do something else, but at the same time creates this real sense of urgency and, and, you know, it's been called hyper competition, which I do agree with. 
 
 

So, um, I started working on preprints when, uh, Ron Vale, who was a professor at my. Ph. D. Institution recruited some people that he knew, you know, being interested in the space, um, Harold Ramos, uh, also among them. And so we, ASAPBio was not originally intended to be an organization. It was simply a meeting, but at this meeting, uh, there was. 
 
 

Interest from funders and advancing the program. And, um, it grew from there to a, uh, into a nonprofit. So I was very fortunate to have this opportunity that I think was [01:01:00] created out of this sort of extracurricular hobby. Um, uh, that was started really by, by people within academia. So I do think that there's an opportunity. For people to do a lot of interesting things within academia, though, I will acknowledge again, the incentives are not necessarily supportive of, of pursuing trying to change the game, the rules of the game that you're playing while while playing the game. So. Um, yeah, I, I think that that was a big factor in, um, you know, ultimately what interests me more, um, the bench work that I was doing and I of course loved, but, you know, or trying to affect these issues, which. Could potentially have a broader impact if successful. And so I think that, you know, for me, I was really fortunate to have that opportunity, but I think there's a lot of opportunities for people who are interested in working in that space. Um, I, I probably a really [01:02:00] compelling alternative would be policy fellowships. Which I know certainly in the United States, there are several of these things that enable early career people to spend time working in government or other agencies. I think that would be also a really good path for people who are thinking more broadly. 
 
 

Benjamin James Kuper-Smith: So what do you do all day? What is your, like, on a practical level, what does that kind of work involve? 
 
 

Jessica Polka: Yeah, so I think what's kind of fun about this is that, you know, as a small organization, there's only two full time staff people at ASAPbio. And so, um, I ended up doing quite a lot of different things that, um, I am certainly not a master of any of these trades, but everything from sort of accounting to communications. the website, um, logistics, things like that. But, but really, um, there's a lot of emailing and a [01:03:00] lot of zoom calls, organizing working groups, planning meetings, writing, editing, and I think trying to organize various campaigns and really talk to others and get their, integrate their input into. Best practice recommendations or, um, or opinion, pieces, etc. 
 
 

So I think that there's a larger variety and that's in a sense that probably the, you know, the most exciting and best part about this, we have a great board who are. You know, I think very willing to, you know, think outside the box, but also keep us a bit grounded in what's going to be functional and productive for researchers who are kind of living in the, the reality of academia on a day to day basis. 
 
 

So, um, yeah, I'm very grateful for that, but, but it is. It is a lot of being like a disembodied presence in front of a computer, and [01:04:00] I, you know, I found that when I moved from bench work to this, that I was really actually missing like pipetting, which might sound 
 
 

like a,  
 
 

Benjamin James Kuper-Smith: can buy  
 
 

Jessica Polka: know, pipetting is, yeah, there you go. Okay. You just solved all my 
 
 

problems. Yeah, I mean, right. I think that. Yeah. Uh, yeah. So, right. I do work from home, which I think is, you know, also like a big change as well. Um, but yeah, um, uh, on the whole, um, there, there's a lot of flexibility. 
 
 

Benjamin James Kuper-Smith: Okay, do you, I mean, like, all jokes aside, do you actually do something to kind of Scratch the itch of like, not, you know, that you, you can't do this kind of like, benchmark now. Do you do something else for that? Or is it just like, just a dream you have to, 
 
 

Jessica Polka: oh my gosh. Yeah, that's such a good question. Yeah. I mean, I think that probably the closest thing that I do to approximate that is [01:05:00] recreational microscopy and specifically bringing microscopy and projected microscopy into environments where it's not necessarily like expected. 
 
 

Benjamin James Kuper-Smith: sorry, what's projective microscopy or, 
 
 

Jessica Polka: Oh, so I'm sorry. I'm sorry. 
 
 

Taking a microscope and hooking up to a projector 
 
 

and allow. And so instead of, you know, instead of people, I do think there is something very magical about looking through some eyepieces, but, um, basically treating a microscope as like a source of av audio visual, um, Enjoyment and entertainment. So, for example, uh, my partner and I last New Year's projected, uh, microscopy at a essentially like electronic music show at a science museum in Florida, 
 
 

which was a lot of fun. Um, so, you know, it's, it's things like that where. There's still it's not just pipetting. It's not just the physical aspect. It's also just [01:06:00] the sense of awe at looking at at life and something about looking at like a single cell, I think, really brings that out that you're getting close to this level where you can kind of look at it as a living thing, but you can also see it as a machine. And that is, I think, really kind of an exciting interface for me. 
 
 

Benjamin James Kuper-Smith: That's really cool. Yeah. I mean, I've, I've done a few episodes about kind of people who try and combine like art and neuroscience. That sounds like you're without even knowing, like, I didn't know that before and that you did this. Um, it sounds like you're also doing a little bit of, a little bit of that. 
 
 

Jessica Polka: Yeah, I'll try. Well, I'd love to do more. It's, um, yeah, it's a lot of fun. 
 
 

Benjamin James Kuper-Smith: A quick question for people who are kind of interested in doing this thing, maybe starting something themselves. I mean, obviously you don't have to. have any funding in the beginning, but I'm just curious. Yeah, I think you mentioned that, you know, at some point SFBio started getting some funding, that kind of stuff. 
 
 

How, how does that kind of process work? [01:07:00] Um, do you just go to like random charities, like scientific charities, and, um, or are there calls for this stuff? Or yeah, how does it work? 
 
 

Jessica Polka: that's such a good, um, that's such a good question. I think, I think that for us, we were very fortunate because We were sort of talking to funders anyway, because their reaction to preprints is incredibly important for the adoption of preprints. And so I think that we were able to engage the funders from this point of view of let's have a conversation about how we collectively are going to work on this problem together. And then I think that the opportunity for funding sort of emerged a bit. From that, but as far as whether there are calls, um, I think that the most significant thing that I'm aware of is, uh, the Chan Zuckerberg initiative does have a, an open science [01:08:00] program that, uh, does give, um, grants and I think that would definitely be something to look into for people who are interested in that space. 
 
 

Benjamin James Kuper-Smith: Okay. Um, Yeah, at the end of each, uh, interview, I have three questions that I'll ask each guest. Uh, first one is, what's a book or paper that you think more people should read? This can be old, new, famous, uh, completely unknown. Uh, just something that you like and think more people should read. 
 
 

Jessica Polka: Um, maybe this is already ubiquitously read by children everywhere, but I definitely had an amazing experience reading Flatland, um, from Edwin Abbott, which was authored in like 1884 as a, uh, as a child. I don't remember exactly how old I was, but I remember being, my mind just being completely blown. By the concept of perception, not being equivalent to reality. And, and I think that, that, you know, maybe [01:09:00] a more practical recommendation for people who are thinking about trying to change culture is influence from Robert Cialdini, which is a book about persuasion and the psychology of persuasion. Written in 1993, originally, I believe, and updated since, and I think that that book has been Incredibly practically useful. I realize I sound, you know, Machiavellian here because on one level I don't want to be Taking advantage or distorting anyone's anyone's decisions In a way that is not constructive and not without their consent, but I do think that encountering the principles that are in this book and understanding how people are making decisions is Illuminating not only in to look at looking into your own behavior, but also thinking about why some of these changes Why certain things are just so recalcitrant and why it's [01:10:00] so difficult to change culture in the first place. 
 
 

Like, for example, there's this emphasis in there about consensus being a huge factor in decision making. And like, what do, what are other people around you doing and what is sort of the accepted practice and And that's something that anybody who wants to make change is going to be running up against. 
 
 

And so, you know, thinking about, again, it's, it's, it's changing the perspective of. A scientist or a human from a logical entity into a imperfectly perceiving primate, um, and I think that's really useful when looking at, you know, how to communicate, but also just looking at my own behavior and decisions. 
 
 

It's helpful to be aware of these things. 
 
 

Benjamin James Kuper-Smith: Okay, I did not, I think it's funny, you've, I think you've given like the most psychological recommendation so far on the show even though you're probably the furthest away from it from the people I've interviewed, so that was kind of, I didn't expect that. [01:11:00] Yes, actually, for once a book I've actually read before, uh, which is nice, but I haven't read Flatlands yet. 
 
 

Jessica Polka: oh, it's so fun. 
 
 

Benjamin James Kuper-Smith: Yeah, someone recommended it to me and then I didn't act on it. Um, but yeah. Second question is something you wish you'd learned sooner. This can be from your work life or your private life. Just something that you think might have improved your life a little bit. And maybe also how you learned it or what you did about it. 
 
 

Yeah. 
 
 

Jessica Polka: about leaving academia, and I think I would express it maybe as that you can have multiple identities that coexist and that goals and. aspirations and values can change over time. Um, I think I really struggled in during my postdoc with the decision of whether to continue along this path toward academia. And, um, I do really, [01:12:00] um, I'm very grateful to have had the opportunity to do something that I'm really passionate about, but I still struggled with the idea of giving up this identity of, um, you know, of being a scientist, right? And it's, um, I think there's a lot of, Pressures in society. I'm thinking back to like writing college admissions essays, or as a scientist or as a researcher, I went to even like workshops about like branding as a scientist, you know, the idea that you create this cohesive sense of identity around who you are and what you're doing. Um, but that doesn't necessarily constrictive when you try to imagine other futures and other possibilities. And so. I do think that everyone contains multitudes, um, so to speak, and I think that if I had understood that a little bit earlier, not just on an academic level, like, of course, I think it's obvious that there's, you know, I think everyone will [01:13:00] agree that, um, going outside of changing plans is like a reasonable thing to do, but it's one thing to say that versus to actually live it and execute it. 
 
 

Benjamin James Kuper-Smith: Yeah, and to realize that you're, that it applies to you. And  
 
 

Jessica Polka: yeah. 
 
 

right, right. 
 
 

right.  
 
 

Benjamin James Kuper-Smith: Yeah. Yeah. Uh, yeah. I mean, my, my final question, maybe you've already kind of answered it partially. I don't know. Um, but the final question is always, uh, any advice for people who are kind of PhD student postdocs or particularly people at that transitionary period. 
 
 

Jessica Polka: Yeah, it's such a good question. Um, uh, I don't know. I think that maybe this could have been an artifact of the particular way that I went through academia, but I don't really feel that outside of writing fellowship applications, there was a lot of structured, Uh, planning. So I think that there, there certainly are, um, individual develop plan, development plans and things like that, that I think are becoming much more widely [01:14:00] used. But I think one thing that I have only realized, um, after leaving academia and moving into another environment, which is also unstructured, but I think there's more of a pressure to actually impose and create structure, um, is sort of the value of like. Deliberately planning and trying to think on a timescale that works for you. Um, for example, I've always found like annual planning to be just like too long for my puny attention span. Um, but I've recently become a fan of quarterly planning and specifically, um, the 12 week year as kind of a framework of thinking about what, what I'm doing on a, on a weekly basis in order to achieve these larger goals. 
 
 

So I don't know.  
 
 

Benjamin James Kuper-Smith: Isn't that book with  
 
 

Jessica Polka: It's a book. It's a book. And I, yeah, and I, I think it's, um, I would say that some of it in the book, um, really [01:15:00] wouldn't be challenging to translate into an environment, which is highly variable, um, where, you know, I think, but I, but the, the simple principle, if I could just express it as like, you come up with a plan, like a quarterly plan and then plan what you're going to do every week to achieve the quarterly goal and then literally check in every week. 
 
 

Right. Right. And that's not like a brilliant, crazy revelation that nobody could have thought of before. But I do think that this weekly cadence is like, you know, there's something it's, I think it's important to connect like weekly to quarterly to broader goals. And yeah. And then, you know, I think there's. I was very fortunate to have some opportunity to have time to explore different things in my postdoc and, and yeah, I, I would encourage everyone who's interested in thinking about, um, perhaps moving outside of a traditional academic career to also find a way to, to do that, um, in a way that, [01:16:00] that, uh, also enables you to, to progress within the lab. 
 
 

Benjamin James Kuper-Smith: hmm. Yeah. Um, I think that those are my questions. So thank you very much. 
 
 

Jessica Polka: Great. Thanks so much, Ben. Nice to chat with you.