Introduction
In modern society, there is little focused intention around helping individuals find the correct problems to solve. Many people live life unaware of their own skills and talents, let alone the value they may bring to themselves and the rest of society. We often praise those who have excelled in their pursuits by devoting their time, resources, and effort to disciplines that provide value to a given community, but we rarely take a critical look at the circumstances that led them to their success. Modern schooling teaches children a fixed curriculum with little to no variation from student to student. This structure is a great fit for some, but for others, likely most, it isn't. Not only are those individuals severely disadvantaged in our society, but our society suffers significantly from those individuals' unfulfilled potential.
This essay will explore how a well-designed problem selection system can play a crucial role in helping individuals make more informed choices by identifying the problems they can solve. I will discuss the key benefits of implementing such a system, the concerns surrounding its implementation and usage, and the importance of addressing these challenges to create a more efficient society.
An Experiment
Suppose an administrator gathers ten students in a classroom with ten problems written on a chalkboard. Each problem has a distinct difficulty concerning an arbitrary subject that each of the ten students is familiar with. Each student must attempt to solve exactly one problem. All students may read each problem, but once a given student has started a problem, that student must attempt to solve it. The administrator has determined that all problems can be solved if and only if each individual were to solve the most difficult problem they are capable of solving. How do the students correctly solve all problems?
Commentary
The reality of the situation above is that the students likely won't solve all problems. Yes, we can assume that the students will be able to assess their abilities reasonably well in this specific scenario, but there is no guarantee that the problem a student is equipped to solve is one that they have seen or solved before. Some students may doubt their abilities and agree to solve a problem at a lower difficulty than their ability allows. Conversely, some students may be overconfident and choose to solve problems above their ability level. Without luck, the students in the classroom can only solve all problems when they are aware of both the difficulty of the problem and their abilities with respect to each problem. However, if either of these is ambiguous, even slightly, the results become much more varied. If the students randomly attempt the problems, there is a one in 3,628,800 probability that all problems will get solved, and, on average, the students would struggle to solve more than five correctly.
Limitations
An obvious limitation of the experiment is that it only addresses a single set of problems. In modern society, there are countless sets of problems. Some problem sets are unique, others depend on each other, and others are subsets of another set or likely many other sets. But even when we expand the experiment to include all problems, the results will remain the same. Without a distinct awareness of both the difficulty requirements of the problem and the abilities of each student with respect to those problems, there is no logical method for assigning students to problems.
Extensions
To expand the experiment further, we can imagine a case in which the ten students cannot solve every problem. In this case, we would likely maximize not only the number of problems the group solves but the difficulty of the solved problems. Or, to expand in another direction, the problems could differ in priority. Then we may neglect some higher-difficulty problems in favor of lower-difficulty problems with a high priority. In another example, we could consider that the students are given time constraints—or are allowed to work indefinitely. Maybe the students are allowed to solve problems in groups. Perhaps individuals can solve several problems—or abstain from solving any. To take it even further, the student's ability to solve a problem could be non-binary.
While these extensions not only complicate the problem, they begin to model the situations presented in our current society much more closely.
The Problem
The experiment above is an abstraction to the reality of life in which a situation is presented with a desirable outcome. In the experiment, it is a problem (the situation) and its solution (the desired outcome). These situations could manifest themselves in many unique ways, such as: seeing the source of inefficiencies (critical analysis), deriving new mathematical formulas (science), constructing a prosthetic hand (engineering), caring for the sick (healthcare), feeding oneself (sustenance), learning an instrument (the arts), conceiving something that has yet to be conceived (creativity), asking questions (curiosity), etc.
We can imagine all labor—even the simplest tasks—as solving some problem. I have attempted to make the experiment as broad as possible to address a new question: What system can we create such that each individual is solving problems that will maximize gain for both the individual and society? As I'm sure you've gathered, we would need a defined measurement for both the problem's difficulty and the individual's ability. Then, our experiment is as simple as writing an algorithm and providing it to a computer to process. So if we hope to solve the scenarios that the experiment models, we must answer several questions:
How do we identify a problem?
How do we assign a measured difficulty to a problem?
How do we determine an individual's capabilities with respect to a problem?
How can we use that information to benefit both the individual and our society?
Before answering these questions, it is essential to understand why solving the problem is necessary. Assuming a solution exists, what would we benefit from achieving it? There are many benefits.
While humans have done great thus far, our society can still become more efficient. There are young adults spending years wandering, hoping to find their passions; adults and elderly individuals who never had the opportunities to pursue the things at which they felt they would excel; and children who are never exposed to the things that they could one day master. We might have experts in fields who could have been pioneers in another or an uninspired office worker who would have made an influential pianist. But the primary effect of overcoming our inefficiencies is not finding the exceptional individuals we might otherwise have missed—it is the subtle increase in individual productivity and satisfaction that comes from excelling at what we do, which will, in turn, allow us to excel even further. Additionally, there are struggles and inefficiencies for those currently working toward solutions for problems that they are well-suited to solve. A new system will remove obstacles and streamline processes so that they cannot only find a solution much faster but also have more time to invest in solving other problems.
We will be more productive with the time we spend and more satisfied with ourselves and the results of our labor. We will eliminate the struggles of modern life at a faster rate and innovate ways to improve our situation. We will cooperate better because we will have set ourselves up to cooperate. And in the process, we will slowly begin to eliminate an uncertainty that has plagued or continues to plague nearly every individual alive today.
The Solution
Unfortunately, we won't solve this problem overnight, and it's evident that to solve the problem in its entirety, we will need societal changes, innovations, and discoveries that have yet to occur. However, we can begin to progress by acknowledging the problem and devoting time and resources to dissecting and solving it.
Based on the analysis of the problem above, there are several steps we must take before implementing a solution. They are as follows:
Determine a unit or units of measurement for the difficulty of a problem.
Determine a process for measuring the difficulty of a problem.
Determine a process for measuring an individual’s ability with respect to a specific problem.
Develop a taxonomy of problems.
Create an interface for the storage and retrieval of these measurements.
Once completed, we can more accurately assess which problems an individual can reasonably solve.
Units of Measurement
An efficient system of problem selection requires a data-driven approach. Most industries have begun to adopt the use of data in their recruiting efforts; however, some have taken a more nuanced approach than others, such as the sports industry. The data required, which can and should be extended to all industries, can be broken down into four key categories:
Biological abilities
Learned competencies
The probability of an individual successfully solving a problem or set of problems
The probability’s confidence level
Many innate aspects of an individual make them unique. Not only do we appear to be different—we are different. While there is still much to discover in this area, an ideal analysis would detail every available biological measurement for an individual. This process will require numerous measurements and combining those measurements into simplified groupings for more efficient analysis. Likewise, many aspects of our nature are learned. With the differences in education and life experience between individuals—there is no guarantee of consistency between those with a similar biological makeup. Therefore we must have a process to assess and analyze an individual's current level of competence, which includes areas such as subject matter knowledge and the ability to use that knowledge. Finally, the most critical data for the individual is the probability of their success and its confidence level. Given that 1) only a finite number of individuals will attempt a problem and 2) we will continually develop new measurements that may be relevant to that problem, we will never be certain of an individual's probability of success. However, over time we can assume that our confidence will improve the more thoroughly we understand a given problem or category of problems. At its finest, the data could be a helpful decision-making tool. For example, assume an individual has a 10% chance of becoming a published neuroscientist but an 85% chance of becoming an elite gymnast. If they were torn between pursuing the two paths, the data might sway their opinion.
A System for Measurement
Although it's not a uniform process, we have measurement systems in place now. Students take admission tests to attend university. Industries have exams that job candidates must pass to receive professional certifications. Companies require resume screenings and an interview process. Sports teams have tryouts and scouting reports. The entertainment industry has auditions and portfolios. However, in most cases, this is only an immediate competency assessment with little to no measurement of an individual's potential or probability of success.
The issue with our system isn't that we don't have processes in place to prevent individuals from working on a problem beyond their abilities. It's that those systems could be more efficient. Someone may spend their entire life working toward a dream they aren't cut out for. Others may give up too soon because they weren't aware of their abilities or their abilities were improperly assessed. With an improved measurement system, we could prevent both of these cases and many others that lead to wasted time and energy. The exact system of measurement is not something that I plan to outline here. However, I will describe a few of its essential aspects.
First, it must be accurate. This requirement should be evident, but it is important to highlight. All data presented must be nothing more than the display of empirical observations. Secondly, it must be transparent. Many of our current systems are opaque and proprietary. They seek, first and foremost, to serve the institution rather than the individual or community. Schools and businesses have reputations to uphold and metrics to achieve. While they hope to develop the individuals they accept and hire, these institutions have no sense of responsibility for those they reject. While some of this may be intentional, it's primarily due to the lack of data available generally. For example, we may not know if there is a biological advantage that successful artists have over the general population. If there were, it might be ludicrous for the average person to expect to be accepted to the Royal College of Art, just as it's likely foolish for a man who is 5' 10" tall to be a center on a professional basketball team. But without data, the art industry has no way to dissuade an average person from building a portfolio and applying anyway. Finally, measurements must be timely. An individual who wants to become a lawyer must attend seven years of post-secondary education and pass a bar exam to become eligible for practice. While the time spent studying is likely valuable for an individual who passes the exam, individuals who cannot pass are left with seven years of experience in a field in which they are not permitted to practice. According to the National Conference of Bar Examiners, some U.S. states report pass rates of less than 60%. While this information is readily available to the public before and could be used by individuals before they begin their journey, the data needs to be more specific. Some individuals likely have a 90% chance of passing, while others have a 10% chance. If relevant biological metrics and competencies are measured before an individual's journey, perhaps we could improve the overall success rate of individuals in our society attaining their goals.
A taxonomy of problems
While, at first, it may be beneficial to limit the scope of the problems that the system measures, an eventual state would require a significant investment in organizing and classifying problems. As time passes, the number of problem categories will be vast and continue growing. The system must be robust enough to expand along with it. In addition to problem categorization, it may be beneficial to include other organizational systems, such as problem themes or shared qualities, that can link problems from a wide range of categories for better analysis. By providing a detailed taxonomy, the system and its users will benefit in many ways, including better organization, system scalability, personalized results, prioritization, progress tracking, identifying underdeveloped problem areas, and streamlining data communication.
A secure storage and retrieval system
Once problems are measured and categorized, and an individual's abilities are determined, we'll need a system capable of not only securely storing that information but allowing individuals to access it efficiently. For a problem selection system to succeed, it must use all available data for its calculations; however, to keep an individual's private information private, it must not be accessible by any other individual. Challenging specifications, such as the one I've outlined above, are becoming increasingly possible with the improvements in computing technology but will still require incredible efforts to build in a secure, private, and scalable manner. Examples of useful modern technologies are distributed ledger technology, distributed file systems, distributed databases, and zero-knowledge proofs. While the technology stack used to create such a system must be extremely well planned, these technologies show promise in achieving a societal scale, private, and secure storage and retrieval system.
Concerns
Implementation and feasibility
The most immediate concern with this solution is the difficulty associated with achieving it. It will take not only decades of effort but decades of communal effort to reach any useful framework, which poses its own set of problems. Technology exists today for each required component, but the resources needed to utilize the technology at a societal scale are immense.
Privacy and data
As with any system, the privacy of an individual's data is of the utmost concern. Improvements in privacy technology will aid in these efforts, but in creating a system protected against malicious actors, we must respect the individual's right to privacy and utilize these technologies to provide a secure system that is private by design. A well-designed problem selection system requires the ability for an individual to upload their own biological and competence-related data for use by all, while that data is unknown and inaccessible by all participants in the system. It is also crucial that a user has direct access to erase all personal data previously uploaded to the system. Finally, as with any technology that uses an individual's personal data, they must be informed of the exact uses of their data before usage.
Adverse effects
Another obvious concern is that the system has unintended adverse effects. As with any system, a poor implementation can be more harmful than good, and while there are paths forward, many paths lead to undesirable ends. In this case, we must be wary of a divergence in the anticipated and eventual reality. The only way to avoid such a divergence is to be clear at each step about what we hope to achieve—committed to finding a solution but not so committed that we are blind to the actual effects of our actions. Some examples of adverse effects are listed below.
Algorithmic bias
As with any algorithmic system of this scale, there is a potential for bias to influence the results. As a simple example, the data retrieved by the user may be sorted so that specific outcomes are more likely to occur than if the data were unsorted. While it is likely impossible to eliminate all bias from a system beyond a certain level of complexity, the system itself should be designed so that any known biases are prevented and that the individual is aware of the potential for bias.
Misuse of information
Problem selection should be an individual's choice. While we can expect that experienced professionals, family members, and friends will weigh in on an individual's decision-making process, it should never be forced upon them. We must also be aware of the societal pressures and stigmas that exist today but would likely be exaggerated in a society that promotes the use of data for everyday decision-making. We don't want to assign an individual to choose a problem because it was identified as the best suited for their abilities, but, at the same time, we don't want to allow our limited perspectives to cloud our judgments on what is best for us, for those around us, and society as a whole. We want to be human, but we don't want our humanity to harm us. The data we find should be used as a tool and resource for educational purposes rather than to dictate which problems individuals will solve.
Segregation
If not designed to be accessible and equitable for all, a new system of problem selection could reinforce social inequalities and lead to an even more classed and segregated society than we have today. There is also the potential that those who don't succeed in solving a problem they have devoted much time toward solving may feel the effects of their failures more prominently—assuming a more significant percentage of the population is succeeding in their endeavors. This result could have adverse effects not only on the individuals who become a minority or disadvantaged—it could also deter other individuals from attempting problems they are not guaranteed to solve. Another avenue for inequality stems from the already existing economic divide. If higher-quality data is available to more privileged individuals, it may allow them to become even wealthier due to their improved ability to solve problems. Even if the data is equivalent for all users, socioeconomic barriers could prevent less privileged individuals from accessing or using the data. Finally, with a poorly written algorithm, the data may inadvertently reinforce the segregation of certain problem areas due to existing trends that only exist due to inefficiencies in our current system. In all cases, some portion of the population is unfairly treated as a direct result of the system, which we should aim to mitigate through intentional design.
Contradictory results
In some systems, the realized outcome is the exact inverse of the intended outcome. Because of the emphasis on data and quantification, the problem selection system described may conflict with necessary aspects of human intuition and creativity. Similarly, emotional friction can reduce the efficacy of a system in many ways. One is lowering an individual's interest in participating in the system. If the system feels rigid and unnatural, stifling our sense of freedom and curiosity, we may lose any benefits from the efficiency we had hoped to gain. Additionally, unexpected discoveries and serendipitous opportunities might be a much larger part of an individual's journey than modern data analytics can model. This reality could manifest through a general reduction in creativity and exploration and a narrowing of social circles that lead to less diverse feedback loops. However, if we struggle with this loss of interest in problem-solving, it may be only a temporary growing pain. We may find that there are optimal problem archetypes for the outcomes we hope to achieve. A long-term problem, such as a career path today, may be best selected with a high probability of success, while shorter-term problems can be more uncertain. There may even be an optimal distribution of problems that a given person can attempt at once. If the system is designed with these concerns in mind, we can lessen the likelihood of these unintended outcomes.
Conclusion
The problem isn't simple, and the solution isn't obvious. Eliminating inefficiencies at such a large scale is an enormous task, and when looking deeply into any problem, many other unnoticed problems begin to appear—all working together to create the pains and struggles that are felt, at some level, every day. However, if we can solve it by creating a robust, accurate, and secure problem selection system, the system could aid in the solution of countless other problems. A society-wide problem selection system has great promise in enhancing individual decision-making, improving problem-solving efficiency, and fostering more effective collaboration. However, the system's successful implementation and usage require addressing privacy concerns, bias, scalability, and many other social considerations. By thoughtfully designing the system with privacy and individual well-being as priorities, we can create a more efficient, well-functioning society where individuals are empowered to tackle problems they are truly capable of solving and make more informed decisions that lead to personal and societal growth. While it may be daunting to introduce even more data into our lives, if used correctly, it could transform the lives of many from good enough to great—from uninspired to exhilarating. So much of what we do is to understand ourselves better. Creating an intuitive and accurate system through socializing problem-solving data in a private and decentralized manner may be the next step in the ever-evolving progression of societal understanding.