Residential Property Characteristics and Assessed Values in West Roxbury
5,802 properties3,481 training1,740 validation581 test
Abstract
Objective. Characterize residential properties by remodel status, lot size, and room count.
Methods. Prepared 5,802 records and conducted descriptive analysis on a fixed 1,740-property validation subset.
Results. Recently remodeled homes averaged 288 sq ft more living area. Lot size and assessed value had a Pearson correlation of 0.55. Six to eight rooms accounted for 75.8% of the sample.
Research Question
Do recently remodeled homes have more living space, do larger lots tend to have higher assessed values, and what room counts are typical?
Why it matters. These comparisons give a reader a baseline for interpreting what makes a property typical or unusual in this sample.
Methods
Prepare. Cleaned column names in pandas and created indicator columns for older and recent remodels, with no remodel as the reference category.
Partition. Used a fixed seed of 5 to reproduce the assignment’s approximately 60% training, 30% validation, and 10% test split.
Compare. On the validation subset, compared mean living area by recent-remodel status, measured the relationship between lot size and assessed value, and counted homes by room count. The assignment also demonstrated standardization. The charts use the original units. Pearson correlation was added for this case study.
Results
Average Living Area by Remodel Status
Validation sample · 1,740 properties
The selected chart uses the same 1,740-property validation subset. Assessed values are expressed in thousands of US dollars in the scatterplot.
Recently remodeled homes averaged 288 sq ft more living area
Among 265 recently remodeled homes, mean living area was about 1,914 sq ft. The other 1,475 homes, combining older remodels and homes with no remodel, averaged about 1,625 sq ft.
Descriptive estimates in the validation sample
Measure
Result
Recently remodeled
1,913.6 sq ft (n = 265)
Older remodel or no remodel
1,625.2 sq ft (n = 1,475)
Mean difference
288.4 sq ft
Lot size and assessed value
Pearson r = 0.550
Six to eight rooms
1,319 of 1,740 (75.8%)
What the Results Mean
Recently remodeled homes averaged 288 more square feet. Lot size and assessed value tended to rise together (Pearson r = 0.55), and about three quarters of homes had six to eight rooms. The remodel comparison describes a difference between groups. Establishing whether remodeling itself explains the difference would require accounting for other property characteristics.
Limitations
The remodel comparison does not control for home age, location, or other property features. The value measure is a tax assessment. These descriptive results do not establish causation or estimate how accurately a model would predict values for new properties.
Objective. Reduce 35 numeric variables while retaining at least 80% of their variance.
Methods. Standardized the numeric inputs for 1,436 Toyota Corolla records and applied principal component analysis.
Results. The first 16 components retained 80.56% of the standardized variance.
Research Question
How many principal components are needed to capture at least 80% of the variation in 35 standardized vehicle features?
Why it matters. A smaller set of inputs can make a dataset easier to explore when measurements overlap or move together. The 80% threshold makes the compression trade-off explicit.
Methods
Summarized the 35 numeric columns across 1,436 records, excluded the three text columns, and standardized the numeric inputs before fitting PCA. No rows were dropped. I then retained enough components to cross the 80% threshold and calculated their scores.
Results
Variance explained by standardized principal components. The horizontal reference in the cumulative view marks the 80% retention rule.
16 of 35 components retained 80.56% of the variance
The first 15 captured 78.36%. The 16th brought the cumulative share above the threshold, reducing the representation by 19 dimensions.
Explained variance at the retention threshold
Measure
Result
First 15 components
78.36%
First 16 components
80.56%
Components retained
16 of 35
What the Results Mean
Sixteen components retained 80.56% of the variation. The components are combinations of the original variables, so this reduces the number of dimensions while changing how they are interpreted. A later prediction study would be needed to tell whether the compressed data preserves the information a model needs.
Limitations
PCA captures variation, and this exercise did not measure downstream prediction accuracy. The Cylinders column was constant in these records and contributed no variance. The component count depends on this dataset and the scaling used.
Objective. Evaluate whether a fixed rule can identify properties labeled higher value.
Methods. Applied an average-room-count cutoff above six to all 506 Boston Housing records and compared predictions with the supplied class labels.
Results. Sensitivity was 97.62%, but the false-positive rate was 59.48%. Accuracy was 50.00%, below the 83.40% majority-class baseline.
Research Question
Can the fixed rule “more than six rooms on average” reliably identify records labeled higher value?
Why it matters. A simple rule can be attractive if it is easy to explain. Its usefulness depends on how often it catches the intended class and how many cases it flags by mistake.
Methods
Classified each of the 506 records as higher value when average rooms exceeded six. Compared the predictions with the supplied binary labels, then calculated the confusion matrix, sensitivity, false-positive rate, and ROC AUC.
Results
Actual class
Predicted lower
Predicted higher
Lower
171
251
Higher
2
82
The dashed diagonal is the chance reference. The marked point represents the fixed six-room rule.
High sensitivity came with many false positives
The rule identified 82 of 84 higher-value records (97.62% sensitivity), but flagged 251 of 422 lower-value records (59.48% false-positive rate). AUC was 0.69070. Accuracy was 50.00%, versus 83.40% for predicting the majority class every time.
Classification performance on 506 records
Measure
Result
Sensitivity
97.62% (82/84)
Specificity
40.52% (171/422)
Accuracy
50.00%
Majority-class accuracy
83.40%
ROC AUC, binary score
0.69070
What the Results Mean
The rule caught 82 of 84 higher-value records while incorrectly flagging 251 of 422 lower-value records. That trade-off may suit a first-pass screen where missing a case is costly. At 50.00% accuracy, it performed worse on overall accuracy than the 83.40% majority-class baseline.
Limitations
Because the assignment used binary predictions as ROC scores, the curve has one interior operating point. The results come from the supplied dataset. No separate test sample or threshold search was used.
Modeling Cereal Ratings With Multiple Linear Regression
Abstract
Objective. Estimate the association of fiber and sugar with cereal ratings and assess predictive fit.
Methods. Fit a two-predictor ordinary least squares model on 51 complete records and evaluated it on 23 held-out records.
Results. Validation R² was 0.839 and validation RMSE was 4.79 rating points on this split.
Research Question
How does each additional gram of fiber or sugar relate to a cereal’s rating when the other ingredient is held fixed? How large are the prediction errors on held-out cereals?
Why it matters. This separates interpreting the ingredients’ estimated associations from checking whether the model gives reasonably close predictions on comparable records.
Methods
Removed three incomplete records and used a fixed seed of 1 to split the remaining 74 into 51 training and 23 validation records. Fit ordinary least squares in scikit-learn and repeated the training fit in statsmodels to compare results.
Results
Validation uses 23 held-out cereals. The residual reference marks zero error. In the comparison view the diagonal identifies equal actual and predicted values.
Validation R² was 0.839
Both libraries yielded a training R² of 0.807. Validation RMSE was 4.79 rating points. Holding the other predictor fixed, the fitted rating changed by +3.03 points per additional gram of fiber and −2.19 points per additional gram of sugar.
Model performance and fitted coefficients
Measure
Result
Training R²
0.807
Validation R²
0.839
Validation RMSE
4.79 rating points
Fiber coefficient
+3.03 points/g
Sugar coefficient
−2.19 points/g
What the Results Mean
The fitted model associated more fiber with higher ratings and more sugar with lower ratings. The 4.79-point validation RMSE summarizes the scale of prediction error, with larger errors weighted more heavily. The associations can inform analysis of these cereals, while a recipe experiment would be needed to estimate the effects of changing ingredients.
Limitations
These are associations from a small dataset with two predictors. Performance may vary with a different split or with cereals outside this dataset. The actual-vs.-predicted chart and validation RMSE were added for the portfolio.
Assess whether fiber and sugars can distinguish cereals rated below the sample median from those rated at or above it. Seven Neighbors Correctly Classified 22 of 23 Validation Cereals.
Research Question
Can fiber and sugar distinguish cereals rated below the dataset median from those at or above it? How does the number of neighbors change that result?
Why it matters. A nearby-cereal approach provides a simple classification baseline. Comparing neighbor counts shows how sensitive the prediction is to how broadly the model looks for similar products.
Methods
Used 74 cereals and a median threshold of 40.25 rating points. Split the data into 51 training and 23 validation records. Standardized predictors using training statistics and compared 15 choices of k.
Results
Validation accuracy by number of neighborsView Exact Values
Validation scores
Number of Neighbors
Accuracy
1
86.96%
2
73.91%
3
86.96%
4
73.91%
5
86.96%
6
78.26%
7
95.65%
8
91.30%
9
95.65%
10
95.65%
11
95.65%
12
86.96%
13
95.65%
14
86.96%
15
95.65%
Confusion matrix · rows are actual classes, columns are predictions
Actual / predicted
Below Median
At or Above Median
Below Median
9
0
At or Above Median
1
13
Seven Neighbors Correctly Classified 22 of 23 Validation Cereals
Validation accuracy was 95.65%. The model classified all nine below-median cereals correctly and missed one of 14 at-or-above-median cereals. Several larger values of k tied at 95.65%.
What the Results Mean
Seven neighbors correctly classified 22 of 23 cereals on the split used to select the setting. Several larger settings tied, so the comparison supports a range of reasonable choices. An independent test would help establish how much of the near-perfect result carries over to new cereals.
Limitations
The full sample defined the median, and the same validation split was used to select k and report accuracy. An independent test would be needed to estimate performance on new cereals.
scikit-learn · Cereals_Mod.csv
Assignment 6 · Regularized Classification
Binary Classification With Logistic Regression
Abstract
Evaluate how well a regularized logistic model predicts the binary termination label in the course dataset. The Model Correctly Classified 12 of 14 Validation Cases.
Research Question
How accurately can a regularized logistic model predict the course dataset’s binary termination label, and which class does it miss?
Why it matters. The small sample makes complex models prone to fitting noise. Looking at errors by class adds context that a single accuracy percentage would hide.
Methods
Used 45 records, with 31 for training and 14 for validation. Indicator-encoded categorical predictors and selected L1 regularization strength by 10-fold cross-validation within the training set.
Results
Confusion matrix · rows are actual classes, columns are predictions
Actual / predicted
Class 0
Class 1
Class 0
7
0
Class 1
2
5
Confusion matrix · rows are actual classes, columns are predictions
Actual / predicted
Class 0
Class 1
Class 0
17
1
Class 1
3
10
The Model Correctly Classified 12 of 14 Validation Cases
Accuracy was 87.10% on training data and 85.71% on validation data. Both validation errors were Class 1 cases predicted as Class 0. The model identified five of seven Class 1 cases.
What the Results Mean
The model got 12 of 14 validation cases right. Both mistakes came from Class 1, where it correctly identified only five of seven. With so few records and demographic inputs in the data, the result is evidence about this split and gives limited information about performance for individuals or subgroups.
Limitations
Each validation prediction changes accuracy by about seven percentage points. The training inputs include sensitive demographic attributes, and subgroup performance was not evaluated. The exercise does not establish suitability for individual decisions.
scikit-learn · LR_dataset.csv · Aggregate validation results recomputed for the portfolio
Assignment 7 · Classification
Ionosphere Signal Classification With Decision Trees
Abstract
Test whether limiting tree depth improves classification of radar returns. The Depth-Five Tree Improved Validation Accuracy to 87.94%.
Research Question
Does limiting the depth of a decision tree improve how it classifies radar returns on the validation split?
Why it matters. An unrestricted tree can follow patterns specific to its training records. A shallower tree may generalize better and is easier to inspect.
Methods
Used 351 records in a 210-record training split and a 141-record validation split. Compared Gini-based classification trees at depths 1–9 and used 10-fold cross-validation within training to compare the depth-limited and unrestricted fits.
Results
Validation accuracy by maximum depthView Exact Values
Validation scores
Maximum Depth
Accuracy
1
75.89%
2
87.23%
3
86.52%
4
87.23%
5
87.94%
6
85.82%
7
85.11%
8
87.94%
9
85.11%
Mean validation-fold accuracy
Depth-five tree88.57%
Unrestricted tree86.19%
Mean accuracy across 10 folds
Model
Training
Validation
Depth five
97.99%
88.57%
Unrestricted
100.00%
86.19%
The Depth-Five Tree Improved Validation Accuracy to 87.94%
The unrestricted tree reached depth nine and scored 85.11%. Mean cross-validation accuracy was 88.57% for the depth-five tree and 86.19% for the unrestricted tree. Depth eight tied depth five on validation.
What the Results Mean
The depth-five tree was about 2.8 percentage points more accurate than the unrestricted tree on the chosen split, and depth eight tied it. The result suggests limiting depth helped here. An independent test would be needed to validate the choice after tuning.
Limitations
The reported validation split informed depth selection. The cross-validation comparison also uses the selected depth, so it is not a nested estimate of the tuning procedure.
scikit-learn · ionosphere.csv
Assignment 8 · Regression
Used-Car Price Prediction With Trees and Random Forests
Abstract
Compare a tuned regression tree and a random forest for estimating Toyota Corolla prices. The Random Forest Reduced Validation RMSE by 13.2%.
Research Question
Does averaging predictions from many trees improve used-car price estimates compared with a single tuned regression tree?
Why it matters. This tests whether the extra complexity of a random forest delivers an improvement on cars held out from model fitting.
Methods
Used the first 1,000 records, with 700 for training and 300 for validation. Indicator-encoded fuel type. The assignment excluded vehicle age and quarterly tax. Tuned the regression tree in two training-only 10-fold grid searches and fit a 300-tree forest.
Results
Validation RMSE · price units
Regression Tree1,948
Random Forest1,691
Training RMSE · price units
Regression Tree1,621
Random Forest636
Regression performance
Model
Training RMSE
Validation RMSE
Validation R²
Regression Tree
1,621.05
1,948.04
0.738
Random Forest
636.16
1,690.98
0.802
The Random Forest Reduced Validation RMSE by 13.2%
Validation RMSE fell from 1,948 to 1,691 in the dataset’s price units. Validation R² rose from 0.738 to 0.802. The forest’s larger gap between training and validation errors also shows why the held-out comparison matters.
What the Results Mean
The forest lowered validation RMSE by 13.2%, from about 1,948 to 1,691 price units, on the same held-out cars. Because RMSE gives greater weight to large misses, the result supports the forest when large errors matter. The comparison is specific to the sample and split.
Limitations
The first 1,000 rows may not represent the entire dataset or today’s car market. These results reflect one split. The portfolio rerun fixes the tree’s random seed for reproducibility.
scikit-learn · ToyotaCorolla.csv
Assignment 9 · Multiclass Classification
Iris Species Classification With Neural Networks
Abstract
Compare two small neural networks for distinguishing three Iris species using flower measurements. Both Networks Correctly Classified All 60 Validation Flowers.
Research Question
Does increasing a hidden layer from two to three nodes improve classification of three Iris species?
Why it matters. Additional nodes increase the network’s capacity. Comparing two small networks tests whether that added capacity helps on this dataset.
Methods
Used 150 records with four measurements each, split into 90 training and 60 validation records. Compared single-hidden-layer networks with two and three nodes, using logistic activation and the L-BFGS solver.
Results
Accuracy
Training98.89%
Validation100.00%
Confusion matrix · rows are actual classes, columns are predictions
Actual / predicted
Setosa
Versicolor
Virginica
Setosa
19
0
0
Versicolor
0
21
0
Virginica
0
0
20
Accuracy
Training96.67%
Validation100.00%
Confusion matrix · rows are actual classes, columns are predictions
Actual / predicted
Setosa
Versicolor
Virginica
Setosa
19
0
0
Versicolor
0
21
0
Virginica
0
0
20
Both Networks Correctly Classified All 60 Validation Flowers
The two-node and three-node networks achieved training accuracies of 98.89% and 96.67%, respectively. Both validation matrices correctly classify all 19 Setosa, 21 Versicolor, and 20 Virginica records.
What the Results Mean
Both networks correctly classified all 60 validation flowers. On this split, the two-node network achieved the same validation result with fewer nodes. The exercise cannot distinguish their expected accuracy on other samples.
Limitations
The two architectures tied on one small validation split. This experiment does not establish that their accuracy would remain perfect on new samples.
scikit-learn · iris.csv
Assignment 10 · Market Basket Analysis and Recommendation
Association Rules and Collaborative Filtering
Abstract
Identify items that appear together in cosmetics baskets and demonstrate how ratings can rank unrated courses. Eye Shadow and Mascara Had the Highest Pairwise Lift.
Research Question
Which cosmetics appear together more often than expected from their individual purchase rates, and can course-rating similarities produce a list of unrated courses for one learner?
Why it matters. Basket associations can suggest bundles worth testing. Rating similarities can give a recommender a starting point when a learner has rated some courses.
Methods
For 1,000 baskets, the assignment used Apriori with minimum support of 20% and minimum confidence of 50%. The figure shows the qualifying one-item-to-one-item rules. A separate exercise used item-based nearest neighbors and cosine similarity to rank unrated courses for one example learner.
Eye Shadow and Mascara Had the Highest Pairwise Lift
The items appeared together in 32.1% of baskets, or 2.36 times the frequency expected under independence. Confidence was 84.25% from eye shadow to mascara and 89.92% in the reverse direction. Spatial was the top course in the notebook example.
What the Results Mean
Eye shadow and mascara appeared together 2.36 times as often as expected under independence, making the pair a candidate for a promotion test. Spatial ranked first in one recommendation example. Neither exercise measured the impact of acting on its output.
Limitations
Basket associations describe purchasing patterns and cannot establish the effect of a promotion. The course example uses observed ratings without a held-out evaluation, so recommendation accuracy is unknown.
Apriori · Surprise KNNBasic · Cosmetics.csv and course-rating assignment files
Assignment 11 · Unsupervised Learning
Airline Loyalty Segmentation With Cluster Analysis
Abstract
Ward clustering identified five groups. The largest contained 2,446 low-activity members. The frequent-flyer group had 130 members and averaged 16.6 flight transactions over 12 months.
Research Question
Can loyalty members be grouped into recognizable profiles using their activity, mileage balances, and reward behavior?
Why it matters. Distinct profiles can help an airline decide which groups to study before testing different offers or messages.
Methods
Standardized loyalty features for 3,999 members. Used Ward hierarchical clustering with Euclidean distance, then fit a five-cluster k-means model for comparison. Interpreted hierarchical profiles using means in the original units.
Results
Hierarchical clusters · loyalty members
1 · High second-card activity43
2 · Low activity2,446
3 · Frequent flyers130
4 · High third-card activity18
5 · High bonus-mile activity1,362
Selected hierarchical cluster means
Segment
Members
12-month flight transactions
Award redemption share
1 · High second-card activity
43
2.21
39.5%
2 · Low activity
2446
0.67
18.4%
3 · Frequent flyers
130
16.64
75.4%
4 · High third-card activity
18
1.33
50.0%
5 · High bonus-mile activity
1362
1.15
66.5%
What the Results Mean
The largest segment contained 2,446 low-activity members, while 130 frequent flyers averaged about 16.6 flight transactions in 12 months. These differences suggest offers to test with different groups. Cluster assignments and offer effectiveness would need separate validation before use.
Limitations
Cluster labels summarize observed profiles. The smallest group contained 18 members. Promotional concepts were illustrative and were not evaluated.
Node C links the triangle to the rest of the network. Its normalized betweenness was 0.667, compared with 0 for node A. Five of the ten possible edges were present.
Research Question
Which nodes are most connected, and which sit on the paths that link otherwise distant parts of this five-node network?
Why it matters. Direct connections and the ability to bridge groups measure different things. Comparing both shows why counting connections alone can miss important positions.
Methods
Represented an undirected network as an adjacency matrix and used NetworkX to compute degree, normalized betweenness, and density.
Results
Five-node undirected network
Degree and normalized betweenness
Node
Degree
Betweenness
A
2
0.000
B
2
0.000
C
3
0.667
D
2
0.500
E
1
0.000
What the Results Mean
A has two direct neighbors but zero normalized betweenness. C has three direct neighbors and lies on about two thirds of the shortest paths between pairs of other nodes. In this example, removing C would disconnect the triangle’s remaining two nodes from D and E.
Limitations
This is a five-node example. Centrality depends on the observed network and does not measure real-world influence directly.
NetworkX · Assignment 12 notebook
Assignment 13 · Text Preprocessing
Preparing Farm Ads for Text Mining
Abstract
The vocabulary fell from 47,513 terms to 43,458 with lemma tokenization, a net reduction of 4,055 terms (8.5%). Both representations retained all 4,143 documents.
Research Question
How does lemma tokenization change the number of features used to represent 4,143 farm advertisements?
Why it matters. Text models often start with many sparse features. Measuring vocabulary size shows how preprocessing changes the input a downstream model receives.
Methods
Constructed term counts and unnormalized TF–IDF features. Repeated both with the assignment’s lemma tokenizer to compare the number of distinct features.
Results
Vocabulary size · distinct terms
Default tokenization47,513
Lemma tokenization43,458
Document–term dimensions
Representation
Documents
Terms
Default counts and TF–IDF
4143
47513
Lemma counts and TF–IDF
4143
43458
What the Results Mean
The lemma tokenizer produced 4,055 fewer distinct terms, an 8.5% reduction, while retaining all documents. The exercise measures the change in feature count. Determining whether the smaller vocabulary improves predictions requires fitting and comparing downstream models.
Limitations
The vocabulary reduction reflects the combined effect of tokenization and lemmatization. The assignment did not compare classification performance across preprocessing choices.
scikit-learn · NLTK · farm-ads.csv
Assignment 14 · Document Classification
Classifying Farm Ads With Latent Semantic Analysis
Abstract
The recorded validation accuracy was 81.01%. The model identified 588 of 649 relevant ads and incorrectly flagged 175 of 594 nonrelevant ads.
Research Question
Can word patterns identify which farm advertisements are relevant, and how often does the model miss relevant ads or flag irrelevant ones?
Why it matters. A relevance filter can help prioritize review. Its usefulness depends on the relative costs of overlooking a relevant ad and reviewing a false alert.
Methods
Tokenized and vectorized 4,143 ads, applied TF–IDF, reduced the representation to 25 components using truncated SVD, and fit a logistic regression model. The saved validation output contains 1,243 predictions.
Results
Validation recall by actual class
Relevant90.6%
Nonrelevant70.5%
Rows are actual classes; columns are predictions
Actual / predicted
Nonrelevant
Relevant
Nonrelevant
419
175
Relevant
61
588
What the Results Mean
The model found 588 of 649 relevant ads and incorrectly flagged 175 of 594 nonrelevant ads. This describes the recorded split. Because the text representation was fitted before the split, preprocessing should be refitted on training data before estimating performance on unseen ads.
Limitations
The vocabulary, TF–IDF weights, and SVD were fitted before splitting. This exposes the representation to validation text. Refit preprocessing on training data before treating the score as an estimate on unseen documents.
scikit-learn · NLTK · farm-ads.csv
Midterm · Model Comparison
Predicting Airline Phone Sales
Abstract
The pruned tree and neural network each reached 86.11% accuracy by predicting no buyers in validation. The full tree identified 46 of 277 buyers (16.61% recall), the highest buyer recall in this comparison.
Research Question
Which classifier identifies the most phone buyers, and does the answer change when we compare buyer recall with overall accuracy?
Why it matters. Only 277 of 1,994 validation records were buyers. A model can score well on overall accuracy by predicting the majority class for everyone, which offers little value for finding buyers.
Methods
Used 2,991 training and 1,994 validation records. Compared logistic regression, full and pruned decision trees, a 500-tree random forest, and a neural network. Calculated validation accuracy and buyer recall from the saved confusion matrices.
Results
Share of 277 actual buyers detected
Logistic Regression0.72%
Full Decision Tree16.61%
Pruned Decision Tree0.00%
Random Forest3.61%
Neural Network0.00%
Share of all 1,994 validation records classified correctly
Logistic Regression85.96%
Full Decision Tree77.28%
Pruned Decision Tree86.11%
Random Forest84.55%
Neural Network86.11%
Validation results
Model
Accuracy
Buyers detected
Buyer recall
Logistic Regression
85.96%
2
0.72%
Full Decision Tree
77.28%
46
16.61%
Pruned Decision Tree
86.11%
0
0.00%
Random Forest
84.55%
10
3.61%
Neural Network
86.11%
0
0.00%
What the Results Mean
The pruned tree and neural network achieved 86.11% accuracy while identifying zero buyers. The full tree found 46 buyers and incorrectly flagged 222 non-buyers. Whether it is worth using depends on the value of finding a buyer relative to the cost of contacting a non-buyer.
Limitations
Class imbalance makes accuracy alone misleading. The full tree also incorrectly flagged 222 non-buyers. Pruning depth selection used the same validation split, so this is a coursework comparison.
scikit-learn · EastWestAirlinesNN.csv
Final Exam · Three-Part Analysis
Applied Analytics Across Clustering, Networks, and Text
Abstract
The pharmaceutical analysis produced four clusters. The network had density 0.333. The review model got 232 of 708 ratings exactly right (32.77%), versus 31.50% for always predicting the most frequent validation rating (10).
Research Question
Can financial measures reveal distinct company profiles, can a small graph identify bridge nodes, and can review text predict exact ratings on a 1–10 scale?
Why it matters. The three tasks show how the evidence changes with the objective. Group profiles describe similarity, centrality describes connections, and classification accuracy measures agreement with known labels.
Methods
Used Ward clustering on nine standardized financial variables for 21 pharmaceutical companies. Analyzed a seven-node undirected graph. For 2,359 drug reviews, used lemma tokenization, TF–IDF, 20-component SVD, and logistic regression to predict rating classes.
Results
Hierarchical cluster sizes · companies
Cluster 14
Cluster 27
Cluster 33
Cluster 47
Seven-node network
Measure
Result
Network density
0.333
Node A degree
3
Node A normalized betweenness
0.567
Node E normalized betweenness
0.367
Rating validation by actual class
Rating
Actual count
Correct predictions
10
223
188
6
30
0
1
100
28
5
45
0
8
69
4
2
31
0
7
42
0
9
122
12
4
23
0
3
23
0
What the Results Mean
The clustering produced four descriptive groups. In the graph, node A appeared on more shortest paths than node E, even though both had three direct connections. The review model’s exact-rating accuracy fell below the benchmark of always choosing the validation set’s most common rating, 10. The saved result therefore gives little support for using the model to predict exact ratings.
Limitations
The company dataset is small and the graph is illustrative. Text preprocessing was fitted before the validation split. The review classification exercise measures agreement with ratings in the dataset.
SciPy · NetworkX · scikit-learn · Final exam notebooks